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The notion of affect does take many forms, and you're right to begin by emphasiz- 
ing that. To get anywhere with the concept, you have to retain the manyness of its 
forms. It's not something that can be reduced to one thing. Mainly, because it's not 
a thing. It's an event, or a dimension of every event. What interests me in the con- 
cept is that if you approach it respecting its variety, you are presented with a field of 
questioning, a problematic field, where the customary divisions that questions about 
subjectivity, becoming, or the political are usually couched in do not apply. 



(Massumi, Of Microperception and Micropolitics, 2009, p. 1) 



The aim of this special issue of the Fibreculture Journal is to address some of the 
contemporary challenges involved in working with affect across disciplines and practices 
that centre on the use of interactive- or digital technologies. The issue has a special focus 
on interaction design, interaction-based art and digital art. The pivotal question, as we 
see it, might be framed roughly like this: How do we explore the "field of questioning" that 
arises when we approach the affective in relation to interaction design, interaction-based 
art and digital art? What is the use of disciplinary goals when the affective has been proven 
most valuable in trans-disciplinary theory? Where do we go from here, that is, how can 
we continue working with the notion of affect, develop it in new theoretical, analytical and 
practical domains? What key concepts would emerge from this continued trajectory and 
how would they feed back onto the theoretical propositions? How would they resonate 
within and with-out existing disciplines, creating novel links and assemblages? 



fibreculturejournal.org Issue 21 - 1 



Editorial: FCJ-21 Exploring Affective Interactions 



With this special issue we present ongoing practices across disciplines that all engage with 
this challenge of working with affect — both analytically and artistically, but always creatively. 
We are especially interested in the way in which changing concepts of affect are taken up 
and modulated within interaction design, interaction-based art and digital art. For example, 
some concepts of affect coming into these areas go beyond the "personal" interaction with 
the technology, indeed beyond (or run beside) many of the assumptions of interaction design, 
including those grounded in phenomenology. They understand affect as an impersonal as 
much as, or even sometimes as opposed to, an intimate dimension of relational capacity. 
As proposed in the work of Deleuze and Guattari, and more recently, in very different ways, 
in the work of Brian Massumi, Patricia T. Clough, Nigel Thrift, and others (see below), affect 
comprises intensities and speeds, in which the living and nonliving, human and nonhuman, 
differentially affect and are affected by each other. Such new understandings of affect have 
consequences for notions of interaction or interactivity, and meet other concepts of affect 
and interaction in ways that challenge basic assumptions about interactive media and digital 
technology in material, processual and experiential terms. 



It is important to underline that this issue of the Fibreculture Journal is not concerned with 
the 'affective turn' per se. Rather, assuming the importance of considering affect across 
a number of disciplines, we are particularly concerned with affect as it is worked with in 
interaction design, interaction-based art and digital art. As Marguerite La Caze and Henry 
Martyn Lloyd clearly demonstrate in their introduction to the Parrhesia issue on 'Philosophy 
and the Affective Turn', studies of affect have a long history within philosophy (La Caze 
& Lloyd, 2011). In their introduction, the 'affective turn' is used to describe a specific 
phenomenon in cultural studies/critical theory in the 90s marking an increased cross- 
disciplinary research interest in pre-cognitive bodily forces, notably in how these forces are 
involved in the construction of human subjectivity, identity and our engagement with other 
people and technology. 



However, after years of intense study we have now reached a point where the analysis of 
the affective has proliferated and spread into a number of disciplines in an attempt to enrich 
the understanding of the pre-individual forces that function on the level of the formation of 
experience — from the micro-perceptual to the macro-political (Gregg & Seigworth, 2010). 
Lisa Blackman and Couze Venn have edited a special issue of Body & Society in which they 
attempt to sketch out the kinds of trans-disciplinary collaboration and engagement enabled 
by the concept of affect as these have emerged across the humanities and the natural, social 
and human sciences (Blackman & Venn, 2010). Indeed, Brian Massumi has described affect 
as a "world-glue" (2000: 187), bringing together different levels of experience and working 
across traditional dichotomies. As such, it seems that affect also has a further role to play 
as a kind of "disciplinary-glue", making disparate practices resonate through the conceptual 
development and practical exploration of affect — and derived concepts, analyses and 
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experimentation. Rather than seeking a unified understanding of what constitutes affect or 
the affective, it will be necessary to develop rigorous approaches across disciplines under an 
affective heading, thus bringing forth the multiplicity of these affective explorations, ensuring 
an enriching dialogue in-between disciplines, and reaching out of an academic context as 
well. 



In an afterword to the above-mentioned special issue on affect published by Body and 
Society, Patricia T. Clough offers interesting ideas about the future of affect studies but 
leaves the question of technology relatively unaddressed (Clough, 2010). Turning towards 
the field of Human-Computer Interaction (HCI), however, a range of technology-oriented 
experiments have been carried out in the name of Affective Computing (e.g. Picard, 1997) 
or Emotional Design (Norman, 2004). These approaches have been criticized within HCI 
for reducing the complexity of the affective in an attempt to make it formalizable and 
structurable in computational and informational terms (Sengers et al., 2002). Recently, 
this informational approach to understanding affect has been countered with what has 
been termed an interactional approach (Boehner et al. 2005; Hook et al., 2008). Here, an 
alternative model of emotion as interaction is introduced, allowing an investigation into how 
interactive systems are experienced as culturally mediated and socially constructed. The 
relation between the affective and emotional remains relatively unexplained, however. All 
this leaves us with a possible space of resonance for many of the findings arising from the 
affect theoretical work done in and around cultural and critical theory. 



Patricia T. Clough's introduction to 'The Affective Turn' from 2007 is explicitly concerned 
with how the 'affective turn is necessary to theorizing the social' (Clough, 2007). Nigel Thrift 
identifies five different schools of affective thinking in 'Turbulent Passions' (Thrift, 2007). 
Interestingly, coming out of psycho-geography and non-representational theory, these 
schools end up mixing together new theoretical assemblages. Brian Massumi offers another 
affective trajectory. In Massumi's work, the philosophical focus moves from Spinoza's 
basic notion of affect as the ability to affect and be affected, through the writings of Gilles 
Deleuze, to other conceptual allies, Gilbert Simondon and Alfred N. Whitehead, at the same 
time making references to work in developmental psychology carried out by Daniel Stern, 
as well as building heavily on William James' notion of radical empiricism. For Massumi the 
notion of the affective has been central for re-conceptualizing the emergence of subjectivity, 
which is not a pre-given entity. One aspect of this is the way in which interactive media 
and technologies may open up new territories for engaging pre-cognitive sensations and 
feelings in bodily experience, in what are sometimes referred to as 'technologies of emergent 
experience' (Markussen, 2005: 2). This re-conceptualization has not only been valuable for 
understanding the aesthetics of interaction as it is continuously explored in interaction-based 
art, digital art, design and architecture (see e.g. Massumi, 1998 & 2007). It has also become 
clear that we need to include the political and ethical in the notion of the aesthetic, which in 
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Guattari's terms leads us to consider the aesthetico-political. Bodies always find themselves 
affected by fields of forces — forces of ideology, techniques and practice — that attune these 
bodies to certain regions of action or potentialities for action (Massumi, 2008: 6). 



With the advent of new media and related technologies, artists and interaction designers are 
offered rich opportunities for exploring the many intersections between affect, sensation and 
action. At the smallest scale we find imaging technologies that allow artists such as Olafur 
Eliasson or Bill Viola to explore microscopically affective layers of sensation, of which we 
may not usually be consciously aware. Turning towards the area known as tactical media, 
one could also find examples such as surveillance technology used subversively in public 
space, either to enhance the affective social attunement between bodies — as in projects by 
Rafael Lozano-Hemmer, Markus Kison or Ben Rubin — or as an instrument for micro-political 
acts of resistance that disrupt existing systems of control and power in order to liberate the 
body and construct counter-publics — as seen most vividly in iSee by The Institute for Applied 
Autonomy or Roderico Dominquez's Transborder Immigration Tool. 



It seems we are at an important point in the exploration of the affective today, one at which 
we are moving from arguing that it is important or even necessary to consider affect, to 
actually working with how affect theory changes different kinds of practices — and not least 
how these practical explorations feed back into and change the theoretical assumptions. This 
is why we are interested in how concepts and meetings of concepts feed into the practices 
that we find in interaction design, interaction-based art and digital art. How do you design 
affectively, for instance? How can we use the insights from and around current explorations 
of affect in a continuously mobilizing and dynamic way, creating new relational events across 
disciplines and practices, feeding into new ways of thinking, doing and acting? If the concept 
of change is so integral to the understanding of affect, how might we actually start "living" 
by it — academically, or in the manner of practice-based research, research-through-design 
or research-creation? What kinds of politics does the concept of affect offer? If, as Brian 
Massumi states, it is possible to talk about the affective as bringing about an expanded 
empirical field in various disciplines, how might we continue an exploratory politics of radical 
change pursued by other than philosophical means? And how do such questions come into 
interaction design, or the more general meeting of technology and the social? 



Affect has been coupled with the notions of interaction and the virtual in an attempt to 
increase understanding of how technology engages and re-distributes human bodies in 
relation to processes, time and change. The need for addressing the question of what affect, 
as a new foundational concept, offers to the understanding of interaction design, interaction- 
based art and digital art seems clearer today than ever. From a variety of intersecting 
backgrounds, the contributions to this issue address this question in experimental, practical, 
and conceptually new ways. 
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We begin with Adam Nash's article, 'Affect and the Medium of Digital Data', in which he 
argues that the notion of affect is critically important for understanding how digital data 
lends itself as a medium and material for creating virtual environments. Too often, Nash's 
argument goes, the term 'virtual' is taken in the sense of virtual reality, a dematerialised 
realm of digital data, which is thought of as being ontologically distinct from material 
reality. While this idea of the virtual certainly was influential in the 1990s in terms of how, 
for instance, the internet was conceived of as a distant cyberspace, Nash is sceptical 
about it becoming 'a signifier for any interaction that is facilitated on a digital network 
and induces affect in the material world, or vice versa'. Drawing attention to his own 
artwork in Second Life, on the internet and elsewhere, Nash gives various examples 
of the way that digital data can be modulated so as to enable non-human and human 
bodies to engage affectively with each other, beyond the digital-material divide. This is 
perhaps experienced most evidently in Trace Aureity, an interactive audiovisual sculpture 
in Second Life where avatars gradually gain a greater degree of autonomy, thereby 
transcending 'the linear mapping between human user and the user's humanoid avatar'. 
Hence, for Nash, there is a need for an integrative ontology, one that views virtual 
environments and more generally 'virtual art' as 'a continuum of force and materiality 
which can be modulated and re-modulated by the artist so new cycles between digital 
networks and material reality can emerge, between non-human and human bodies.' And 
the notion of affect is a core concept in this ontology. 



The idea that the notion of affect is key for understanding how digital technology gives 
artists access to work with hitherto unexplored forms of interaction is also central in the 
second essay, Affect and Care in Intimate Transactions', by Lone Bertelsen. Bertelsen 
shifts the focus of attention from ontology to ethics insofar as she discusses how ethics 
may be rooted in the way that bodies mutually affect and are affected by one another. 
In her article, she examines Intimate Transactions, an immersive interactive installation 
where participants situated at two distant locations can experience intimate transaction. 
Each participant uses a physical interface called a 'Bodyshelf. By gently moving their 
bodies on the Bodyshelf they instigate intimate transactions, which influence an evolving 
world of non-human creatures. As these creatures meet in the screen-world, they can 
'move together as one semi-merged avatar'. At the same time, vibrations onto the lower 
back and lower abdomen of the participant's bodies are activated. For Bertelsen, this can 
be seen as an instance of 'co-affective collaboration', which focuses on 'trans-subjective 
collaboration and a logic of affects'. This leads to a form of experience which cannot be 
adequately accounted for with the notion of interactivity. This is because, while the notion 
of interactivity presupposes the individual and the subjective as pre-existing categories, 
co-affective collaboration takes place at a pre-individual and trans-subjective level. In 
fact, the ultimate, but implicit conclusion of Bertelsen's argument seems to be that, in the 
vocabulary of media art theory, interactivity must be supplemented with trans-activity as 
a new foundational concept. For years critical thinking on digital art has put its trust into 
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the explanatory strength of interactivity. Bertelsen suggests that a re-thinking is necessary 
and that affect is a promising starting point for future work. 



Susan Kozel's 'AffeXity: Performing Affect with Augmented Reality' situates us immediately 
in the middle of a transdisciplinary inquiry into affect in cities and a-fixity as an urban 
condition. In the article, Kozel unfolds the affective explorations carried out in an ongoing 
interaction design project, AffeXity, experimenting with artistic practices from dance 
improvisation, video shooting, digital image editing to sound composition, combined with 
the daily practices of moving through a city and using mobile devices. Kozel writes at the 
intersection between conceptual coherence and artistic direction in an attempt to bring to 
life the way that working with affect simultaneously modifies both theory and practice, in 
a writing style that brings to the surface the affective explorations involved. Thinking with 
and through affect theory, digital media and social choreographies, Kozel develops a notion 
of performance triangulated across bodily movement, emergence and shimmering. Starting 
from the basic assumption that designing affectively and designing for affect are two 
different things, Kozel proposes a range of affective sensibilities. Her work straddles practical 
and theoretical activities because 'it is used in the process of generating the movement and 
media at the same time, as it is a way of engaging with theories of affect — it is a method for 
generating artistic and theoretical content.' 



In the fourth article Mark Gawne emphasises the lack of awareness inside compositionist 
analyses of the way in which affective technologies are used to organize labour in the 
post-Fordist condition. Theorists of affective labor such as Hardt and Negri have been 
successful in demonstrating that in 'the passage to post-Fordism, the labour of producing 
affects, communication, knowledge, the creation and maintenance of relationships and the 
cultivation of attention emerge as key economic terrains'. Gawne argues however that while 
these theorists have identified the need to consider the problem of immaterial production, 
they have less to offer in critiquing how technology is used and misused 'to subordinate user 
affect to the imperatives of capitalist valorization'. For over a decade, Human-Computer 
Interaction (HCI) has focused on developing 'technologies that aim to sense, recognize 
and modulate user affect'. However, through his critical and enlightening examination 
of recent developments in affective HCI, Gawne identifies a blind spot insofar as HCI 
research seem reluctant to consider the impact that these technologies have on the bodies 
involved. Gawne's contribution consists in integrating a discussion of affective HCI with the 
perspective of compositionist analysis. In so doing, he remedies the inherent limitations 
of both fields. The heuristic value of this endeavor becomes evident in Gawne's analysis 
of smile-scan, a technology developed by Japanese company OMRON for the purpose of 
measuring the face expressions of workers within workplaces. 
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The four articles are followed by two conversations. The first of these is 'Multimedia Mixing 
and Real-time Collaboration: Interview with Sher Doruff about the development and use of 
KeyWorx, the Translocal and Polyrhythmic Diagrams.' This conversation recounts a largely 
undocumented chapter in the history of the media art and technology nexus. While books 
on media art flourish that contain accounts of the ways that artists have exploited existing 
technologies coming out of research and innovation in industry for artistic purposes, 
the conversation with Sher Doruff is remarkable. It reveals the way in which artistic 
experiments themselves have launched new networking technologies which in many ways 
anticipate more recent developments in social networks. In a personal conversation with 
Andrew Murphie, Doruff takes the reader behind the performance scene in the 80s and 90s, 
where, as an artist, working in New York and Amsterdam, she collaborated with various 
performers, dancers, musicians, and programmers in developing Keystroke. Keystroke, 
or Keyworx as it was later re-named, is a virtual studio environment, which in 97 gave 
artists and performers the opportunity to engage in translocal, real-time collaborative 
performances. That is in 97! Even though the notion of affect may not appear to be placed 
at centre stage, the affective is felt to be present as a form of co-existence among the 
artists and performers, one for which the Keystroke technology is designed. In translocal 
performances an intense 'synchronous interaction' emerged between the performing 
bodies. Performers always need to find a rhythm together. But translocal performance, 
where bodies are separated, is different, or as Doruff explains: 'You don't have perceivable 
body language between you. So you have to find other ways to find that kind of 
synchronisation. It's incredibly intense and affective'. This affective synchronisation is not 
only being explored in many of Doruff's own art projects, which are presented throughout 
the conversation; it is also closely aligned to the co-affective experiences discussed in the 
earlier essays by Nash and Bertelsen. 



In 'Entertaining the environment', a conversation between Andrew Goodman and Erin 
Manning, the affective is conceptualized in terms of 'relation', which is counterposed to 
'interactivity'. 'Interactivity' or 'interactive art work' are two concepts used by Goodman 
and Manning to describe artwork, where the 'art event' is drowned out by the 'technology 
event'. Or, as Goodman says, works of art 'that seem invested in a demonstration of 
technology's capabilities (and/or the artist's technological skills)'. Relational art, on the 
other hand, is more interested in what art can do and not just what technology can do. 
It activates experiences that do not place the viewer at the centre of the experience, but 
invites her instead to participate in creating events, and this subverts the hierarchy of 
subject and object. This disruptive aesthetic effect or 'tweaking of experience' is not a result 
of technology use, but can be achieved by simple means and techniques such as those 
introduced by the conceptual art movement in the 1960s and 19070s. By tracing the roots 
of relational art back to conceptual art, Goodman and Manning point towards affective 
possibilities 'at the fringes of technology'. 
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Introduction 



This paper attempts a technical analysis of the medium of digital data to establish how affect 
may emerge in that medium. Two central questions here are, first, whether it is possible for two 
immanently digital entities to establish an affect cycle with each other, and, second, how this 
relates to affect cycles established between digital data and non-digital entities? It should be 
possible to build artworks that can test certain of their own intrinsic properties in both these 
respects. The author had a hand in creating some such artworks and these are examined later 
in this paper [1]. 



The constant movement of data in a process of modulation, demodulation and remodulation is 
one of the defining characteristics of the digital medium. Regardless of the final display charac- 
teristics and potential interactions of any given digital bit, it is constituted through a constant 
process of digital data modulation. Modulation is used here to mean the process of changing 
some phenomenon from one register into another, for the purpose of storage, transmission 
and display. The term also resonates with Deleuze's (1992: 3) sense of modulation as 'like a 
self-deforming cast that will continuously change from one moment to the other, or like a sieve 
whose mesh will transmute from point to point,' as well as the musical sense of changing key, 
the electronic sense of changing a signal with another signal or the social sense of changing 
one's tone of speech according to listener or circumstances. All these share the characteristics 
of both intentionality and change. Deleuze (2003: 84) also uses the term in relation to Francis 
Bacon's use of colour, to describe an intentional change of relationship. Steven Goodman (2009: 
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xiv, xvi, xix) uses modulation in a similar sense when talking about the relationship between 
sound, affect and vibration. 




Figure 1: Screenshot from Autoscopia by Justin Clemens, Christopher 
Dodds, Adam Nash, 2009-present: generated portrait of Adam Nash. 
Image and permissions provided by Adam Nash. 



There are other words that can be used to describe the process of conversion to and from, 
and transformations performed by, digital data (en/decoding, conversion, transduction, etc). 
However, I prefer modulation precisely because it maintains an overtone of change. This 
overtone facilitates a constant challenge to the often unexamined assumption that digital 
data is somehow ontologically endowed with the special power to make exact copies of 
anything, that to digitise something is to somehow capture it free of interference from the 
capturing medium. In fact, the digitising of anything involves a complex series of protocol 
negotiations and these negotiations create an excess of data. This excess of data is a new 
object created in excess of the object being digitised, comprising the data from which a 
simulation of its semantic source may be constructed, a set of data about the data, and the 
object created by the synthesis of these two sets of data plus the new object's knowledge of 
itself. Goodman (2009: 121) also acknowledges the excess created through digitisation, and 
wonders about the implications for affect in such an excess. 
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Assemblages involving digitisation are also constantly and recursively deepened. As just 
mentioned, each step in digital processes relies on a protocol pre-agreed (usually by 
humans). These protocols determine how to proceed. References to these protocols and 
minimal descriptions of the nature of the data's semantic source constitute some of the 
excess constantly created within processes of digitization. However, it is important to note 
that a digital file does not usually contain instructions on how to decode it in order to 
reconstruct a copy of its semantic source. Rather, digital files simply note which protocols 
are required to do so, in order that the system can employ the appropriate software to 
attempt a decoding. Such software is itself subject to the same process and so on. The point 
is that digital data is essentially formless and plastic. It requires an intentionality [2] external 
to itself in order to be reconstituted as a copy of its semantic source. This is why I use the 
terms modulation, demodulation, and remodulation, to keep in mind the intentional and 
transformative nature of the operations performed on, within and by digital data. 




Figure 2: Screenshot from Autoscopia by Justin Clemens, Christopher 
Dodds, Adam Nash, 2009-present: generated portrait of Marilyn Monroe. 
Image and permissions provided by Adam Nash. 



In this article, I am specifically interested in the modulation process that occurs between 
data-as-data and data-as-display. Here, display does not necessarily mean visual display, 
but, rather, any mode by which the data can be perceived, which may be visual, aural, 
textual, physical, relational, whatever. Such display may not be restricted to human 
perception, which I will discuss later. When it comes to display, modulation is a constantly 
occurring, crucial interaction that, at least to some degree, defines the nature of perceptions 
and therefore the work done in and through the medium with regard to display. Yet, here 
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as elsewhere, at the level of data, modulation is a process that effaces its own input. This 
provokes further questions. What is the ontological status of data in such contexts? What 
are the consequences of this constant modulation from one register to another, within 
data, and through data of non-digital material events such as display? What is the status of 
artworks produced within such a medium? How does data modulation influence the nature 
of the interactions and perceptions, or the escape and capture of affect? 



Virtual environments are post-convergent 



In order to reflect adequately on the new affective relations enabled by the modulation 
and remodulation of digital data, we can try to examine the intrinsic qualities of virtual 
environments. Firstly, virtual environments can be seen as post-convergent. This term builds 
on Henry Jenkins' notion of 'convergence', and designates the phase in the development of 
a new medium when it recognises itself as such, when practitioners begin operating within 
the medium to explore its intrinsic qualities - as opposed to mere expressions of its content, 
that is, prior media - to create work that is only possible in the new medium. (Clemens & 
Nash, 2010; Deleuze & Guattari, 2004: 261; Elias, 2011: 199; Jenkins, 2006: 18) Networked, 
digital, virtual environments exemplify contemporary post-convergent practice. The art of 
virtual environments is not simply art that relies on digital technologies. Rather, it is art 
that intrinsically operates in the excess that is created in the digital medium. This technical 
excess is also the excess that is necessarily created in the emergence of a new medium. A 
new medium contains all prior media as content, and thus is convergent. Yet it is also the 
container, in excess of these contents. The sum that is greater than its parts, and thus is 
post-convergent. (McLuhan, 2001: 8-9) 



One of the problems of understanding virtual environments as post-convergent is 
vocabulary. Vocabulary is always challenging when discussing any new form of artistic 
endeavour. At crucial defining moments an unsatisfactory choice arises between retrofitting 
existing vocabulary at the risk of forcing a genuinely new concept into the expectations 
created by the existing vocabulary, thereby losing the very novelty whose expression was 
attempted, or creating neologisms that potentially confuse more than elucidate. This is 
particularly true of the term 'virtual', since it is used differently in mainstream culture, media 
studies, affect theory and philosophy. The current casual mainstream usage, meaning 
some kind of affect-producing interaction that takes place on the digital network, reflects a 
growing acceptance of reality as comprising a symbiotic combination of online and material 
experience - an acceptance that has perhaps always existed, even if implicitly, but is brought 
into explicit focus by the growth of digital networks as a significant cultural force. As Larissa 
Hjorth (2011: 65) says, as 'the internet has developed to become an integral part of everyday 
life globally, the online has evolved into a complex set of networks and communities that 
have challenged traditional notions of online/offline relations.' 
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In media studies, the term 'virtual' is sometimes (mistakenly) seen as part of a discussion 
that took place in the 1990s and is therefore somewhat old-hat for contemporary debates. 
This is more indicative of contemporary media studies' habit of mistaking the naming of 
something for the understanding of something, than an indication of the usefulness of the 
term. The vocabulary is further confused in media studies through the apparent conflation 
- or at least confused use - of two ostensibly very different concepts: 'virtual' as in 'virtual 
reality' and 'virtual' as in Gilles Deleuze's use of the term. Deleuze's 'virtual' is an important, 
nuanced and multifarious concept that is not easily reducible to a single definition, especially 
given Deleuze's use of the term in different situations over the course of his career. However, 
we can crudely characterise Deleuze's 'virtual' as being 'the characteristic state of Ideas' 
(2004: 263), where 'an Idea is a "complex theme", an internal multiplicity - in other words, 
a system of multiple, non-localisable connections between differential elements which is 
incarnated in real relations and actual terms.' (2004: 231) Deleuze's term is strongly informed 
by Bergson's intuitionism, and as such is often mistakenly taken to simply mean 'possible'. 
However, Deleuze is very clear on the distinction between the possible and the virtual: '[t] 
he possible is opposed to the real... [b]y contrast, the virtual is not opposed to the real; 
it possesses a full reality by itself... [w]hat difference can there be between the existent 
and the non-existent if the non-existent is already possible!?]' (2004: 263) Cleared of this 
confusion of the possible and the virtual, Deleuze's virtual can offer interesting insights into 
how affective experiences can emerge in the digital (or at all), and Anna Munster (2006: 
92) has convincingly shown that 'one of the wider aesthetic implications of processes of 
digitization has been to impinge upon and give a certain form to manifestations of the 
virtual.' Furthermore, in one of the key texts of affect theory, Parables for the Virtual, Brian 
Massumi (2002: 35) locates affect precisely in the two-sided 'simultaneous participation of 
the virtual in the actual and the actual in the virtual.' 



As for the 90s usage of the term 'virtual reality', it was a contemporaneously useful term 
to capture what was a novel expression of the intersection of material and conceptual 
reality (or, of data-as-display and data-as-data), heavily informed by the cyberpunk fiction 
of William Gibson, Neal Stephenson and other authors. (Hjorth, 2011: 65; Pesce, 1995) 
The term can retrospectively be seen as reflecting and reacting to the virtualisation of 
capital that occurred in the 1980s, as capital was brought into digital networks. (Attali, 
2009: 86) Simultaneously fascinated and repulsed by the implications of this virtualisation, 
these authors - and subsequently mainstream media - experimented thoughtfully with 
dematerialisation and the nature of the symbiotic affect system that virtualisation implied 
(I see this as "symbiotic" in the mutual dependence of data and the non-digital). As I said 
above, mainstream media simplified the term (dropping the "reality") so that it became 
a signifier for any interaction that is facilitated on a digital network and induces affect 
in the material world, or vice versa. Much of this usage also has roots in an arguably 
religious reading of virtuality. (Wertheim, 1999: 20). Yet as the lived experience of the nexus 
between material and virtual reality has become more common in contemporary society, 
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the term "virtual" has come to designate something closer to Deleuze's virtual, that is, it 
describes one part of a relation between the virtual and actual. Put differently, this can be 
characterized as what Anna Munster (2006: 90) calls 'a differentiated continuum of force and 
materiality.' 



To quickly discuss another relevant term, I suggest that 'digital art' is, at this juncture, not 
a useful definition for the artworks I am describing, because the term has been co-opted 
by pre-digital art forms to designate simply a tool in the creation of existing artforms. In 
fact, this is the colonising nature of all artforms, since they are incapable of conceiving of 
a world that is not contained within their parameters. Naturally, this colonisation results 
in a new assemblage where the existing artform operates in dialogue with the newly 
adopted digital tools, creating a kind of simulated version of the artform. This retroactively 
uncovers previously unknown potentials and tendencies of that artform. Nonetheless, the 
existing artform's intention remains the absorption of digital data into its established value 
structure. For example, in the digital medium, an oscillator is just another object defined, 
analogically, by its intended behaviour, as is a channel, and there are an infinite number 
of them available, neither more nor less important than a triangle, a colour, human input 
or a set of stock market data. Therefore, to talk of 'multiple oscillators' or 'two channel 




Figure 3: Screenshot from Autoscopia by Justin Clemens, Christopher 
Dodds, Adam Nash, 2009-present: generated portrait of Albert Einstein. 
Image and permissions provided by Adam Nash. 
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dvd' is meaningless except as an obfuscatory device to establish the authority of the artist. 
Such a device relies on pre-digital notions of restricted access to means of production and 
distribution as a method for establishing uncontestable value, entirely rooted in an artificial 
rarity model unrelated to the work itself. It is in order to avoid this absorption of digital data 
into the value structures of pre-digital artforms that I use the term 'virtual art'. This refers to 
art that intrinsically uses digital data as its medium, but in the sense that digital modulation 
allows for Munster's 'differentiated continuum of force and materiality.' 



Ontology of Digital Data 




When you give people too much information, they instantly resort to pattern 
recognition to structure the experience. The work of the artist is to find patterns. - 
Marshall McLuhan (Coupland, 2009) 
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Digital data is formless, plastic and leveling. Stored as binary bits, it has no form as 
such. As Justin Clemens and I have written (2010), 'Data is data. Data is absolutely not 
a phenomenological thing. It cannot be experienced as such, like Aristotelian prime 
matter. Unlike Aristotelian prime matter, however, we can manipulate data with ease.' The 
fundamentally plastic nature of digital data is what allows us to manipulate it, but until 
we do manipulate it - until we modulate it into some kind of display register - any set of 
digital data is indistinguishable from any other set of digital data, until modulated into a 
display register, and this is the leveling nature of digital data. All information is reduced to 
an indistinguishable set of binary bits. To illustrate this, consider a digital image, such as 
may have been taken by a digital camera of a material scene. Once this visual information 
is stored as digital data, it can then be opened in, for example, a sound editing program and 
played as sound. It could equally be used as input to determine a height-map in a realtime 
3D environment. The point is that once it is stored as digital data, it loses any determining 
connection with its semantic source. Therefore, as I said above, parameters must be 
rigorously established that govern how any given digital data is de- and re-modulated. The 
notion of protocols or standardised processes that abound in the contemporary technical 
sphere (such as govern the internet, image compression, audio reproduction and so on) are 
expressions of this codification of parameters - both sides of a modulation exchange agree 
to adhere to a set of parameters in order that the intended result is achieved. Naturally, once 
protocols are required, questions of intentionality, ideology and cultural convention arise. 



In this way, the decision to remodulate the data into a display register that somehow 
resembles its semantic source (for example, to display the data from the digital camera 
image as an image) can be considered a creative act, an intentional act of representation. 
While the digital camera example may be conceptually straightforward, consider the case 
of using motion capture data to drive the animation of a humanoid figure in a realtime 3D 
environment. It is simply not possible to recreate, in a virtual environment, the movements 
of a human in material space, therefore intentional decisions must be made as to how 
best to simulate the appearance of human movement. It is sometimes easy to forget that 
simulations (such as are to be seen everywhere these days - weather simulations, physics 
simulations, market simulations, etc) are actually simplified representations, because 
modulation is necessarily modulation into another register, a register in which the originating 
source does not exist. 



Further, virtual environments may be composed entirely of digital data. That is, the entities 
that constitute a virtual work may themselves originate from within the virtual environment 
without reference to an associated entity in the material world. They are composed of data 
and have their provenance in data. In other words, virtual environments can be data mined 
for input back into themselves. Such intrinsically virtual entities raise interesting questions. 
They are purely data-as-data and there may be no protocol, or social convention, that 
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might determine how to modulate them into display. In this context, the establishment of 
parameter frameworks reveals itself as one of the fundamental process, or acts, that the 
artist must engage with when creating virtual environments. Such a process requires a 
conscious acknowledgement of the difference between data-as-data and data-as-display. 
Can the site of modulation between data-as-data and data-as-display be seen as a site for 
the emergence of affect, a site where entities defined by their potential for interaction may 
partially capture, and witness the escape of, affect? By investigating this site of modulation 
in such a manner, connections are opened with the concept of biomedia, seen from the 
perspective of affect theory. Eugene Thacker, who coined the term biomedia as the title of 
his 2004 book, says in a 2010 essay, also entitled Biomedia (123), that "with biomedia, we 
do not have a split between biology and information, life and code, nature and artifice". He 
goes on to say (2010: 126) that "[b]iomedia present a view not merely of biological life as 
information, but of biological life that is life precisely because it is information." 



The work Autoscopia was a collaboration between myself, Christopher Dodds and Justin 
Clemens, commissioned by the National Portrait Gallery of Australia for their Doppelganger 
exhibition in 2009. Autoscopia was an attempt to explore the affective cycle established 
between the material and the networked self. The work creates search-based composite 
portraits, allowing users to enter names in order to create virtual portraits based on internet 
searches. Both the image and text components of these portraits are composited from the 
results of web-based searches on the inputted name. The searches exploit the usual sources 
like Google, Facebook, Twitter, Linkedln, and so on, as well as more insidiously invasive (but 
nonetheless publicly available) search engines specializing in background checks and public 
record searches. The results manifest as web pages containing the dynamically composited 
image and text, as well as audiovisual sculptures dynamically generated in Second Life. 
The Second Life component closed at the end of 2010, but the web portraits continue to 
grow, all the while 'tweeting' their existence on Twitter, recursively feeding themselves back 
into the results of future searches. The work has been continuously running online long 
enough now that Google will actually return the Autoscopia page for certain names as the 
top ranking result. Within the virtual environment of the world wide web, this represents an 
emergent privileging of the affective power of digital entities over the material entities (that 
is, people) that putatively caused the existence of the digital entity in the first place. In other 
words, the trace left by the digital entity may have more power, in the virtual world, than 
the trace of its associated material entity. This raises very interesting questions about the 
nature of the affective power and instrumentality of the trace of immanently digital entities, 
that is, entities that do not have an associated material entity or entities that emerge from 
an associated material entity but take on contingent agencies immanent to the virtual 
environment. I will discuss this later in relation to my work One, Another. 
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Autoscopia also brings into relief the process of modulation between data-as-data and 
data-as-display. Data-as-data is mined by Autoscopia from all over the internet. This 
'mining' involves a constant process of modulating the data-as-data into data-as-display, 
not according to the original semantic intentions that led to the creation of that specific 
set of data-as-data, but according to the framework of parameters that can be said to 
constitute the artwork of Autoscopia. This modulated and remodulated data is then stored 
by Autoscopia as data-as-data ready to be modulated into data-as-display, contingently 
upon interaction with an end-user. 



Affect and digital data 



My notion would be, that anything which possesses any sort of power to affect 
another, or to be affected by another, if only for a single moment, however trifling the 
cause and however slight the effect, has real existence; and I hold that the definition 
of being is simply power. - Plato, Sophist, 247e 

According to Deleuze (1988: 124-126), if we take a Spinozan view of affect, then we do not 
need to be concerned with the difference between nature and artifice, and can concentrate 
only on bodies' capacities for affecting and being affected, defined by their 'compositions of 
relations.' For Spinoza, a body can be anything that is capable of affecting or being affected, 
and he is concerned with an immanent plane of specific encounters, in opposition to a 
transcendent, anthropocentric morality. Deleuze says that 'a body can be anything; it can 
be an animal, a body of sounds, a mind or an idea.' Highlighting the musical implications of 
the word composition, Deleuze goes on to portray Spinoza's Ethics as both describing and 
itself constituting a musical composition (Deleuze, 1988: 127). This is obviously going to 
appeal to me as an artist working with digital code, especially an artist with a background 
in performance and composition, because it allows us to examine the affective abilities of 
emergent digital entities - on each other, on their virtual world, and on human interactors. 
It also allows us to consider the assemblage established between virtual and material 
worlds as a world (or composition) itself. Not only that, but since I also believe that music/ 
sound practice in the 20th century offers some useful approaches for artists working with 
the medium of digital data, it is very useful to consider the affective world (or 'the plane of 
immanence') in terms of musical composition (Deleuze, 1988: 128). 



Some contemporary affect theorists define affect as autonomous from consciousness, 
language and emotion (Clough, 2010: 209). Emotional response is subsequently seen as a 
retroactive narration of affective response. Massumi, for example, says that affect is pre- 
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individual and pertains to a different order than emotion, and that affect can be extended to 
'any or every level, providing that the uniqueness of its functioning on that level is taken into 
account.' (Massumi, 2002: 27-37) Other critics have highlighted a potential paradoxical return, 
inherent in these claims, to a mind/body dualism that would run counter to the Spinozan 
monism that was supposed to have prompted the claims in the first place (Leys, 2011: 434- 
472). However, as an artist working in the medium of digital data, it is useful to be able to 
examine the affective power of digital entities separately from the emotional response it may 
elicit in human users of the work, and then to consider the ways in which these separate 
phenomena knit together to form a complex feedback system that constitutes a virtual 
artwork in its interaction. Such a system may then be examined for its emergent ability to 
constitute a site for the capture and escape of affect. As an artist, this approach makes sense 
to me, allowing a study of compositional relations, not forms, and the traces that emerge. 




Figure 5: Screenshot from Trace Aureity by Adam Nash, 2008, 
commissioned by Networked Music Review for Turbulence. 
Image and permissions provided by Adam Nash. 



My work Trace Aureity, a work in Second Life, attempts to examine the nature and potential of 
such traces by establishing a network of relationships between the user and the environment, 
not only by investing the virtual space itself with interactive audiovisual properties, but also 
by spawning moving digital agents in order that different traces are inscribed within the 
environment by the users' interaction with it (Nash, 2008). I have already stressed that one 
of the intrinsic features of virtual artwork is its capacity to create entities that originate in 
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the virtual environment, and are composed entirely of data. Trace Aureity explores the 
affective and relational possibilities this opens up. The digital agents within the work are 
spawned in response to user proximity, but once spawned, begin to automatically determine 
and enact paths through the work. These agents have the same interactive effect on the 
work as the user, that is, an agent moving through any particular element of the work will 
cause that element to react in the same way it would were the user to move through it, 
since at the level of digital data there is no meaningful distinction between them. In this 
way, the trace that the user inscribes within the space of the work becomes a branching 
one, and somewhat autonomically aleatoric. This trace always maintains a relationship 
with the user's path through the work, because agents will only be spawned from the 
user's position. Yet this initially strong relationship between the user's and the agent's 
paths becomes weaker over time as the agent gains greater degrees of autonomy from its 
provenance. At the same time, since the agents always spawn initially in response to the 
user's movements, a non-linear network of relationships is established, in which the semi- 
autonomous behaviour of the agents can never be said to be completely independent of 
the user. In other words, it is always the user that is playing the work (even if not totally in 
control of the work). In this way, Trace Aureity can be seen as an example of a multi-sited, 
or non-linear, avatar that transcends the linear mapping between human user and the user's 
humanoid avatar. The work thus represents an attempt at transcending the human-avatar 
metaphor. The user is invited to navigate the work in a reflective manner, to experience as 
many sites of interactive relationship as possible, to play the space in a virtuosic sense as a 
result of removing all tendencies toward a forward or linear navigation or interaction model. 
For the games scholar Bernadette Flynn, this kind of reflective, or contemplative, navigation 
represents a 'central organizing device' through which the agency of the user 'enables the 
experience of profound ideas and different modes of consciousness.' [39] Of course, it is 
important to remember that the register of such experience arises after the pre-individual 
fact of the autonomic establishment of an affect cycle and represents what Massumi (2002: 
35) identifies as the 'capture and closure of affect.' 



In Trace Aureity, there is an implicit expectation that the user will exercise their virtuosic 
ability to navigate the work, manipulating the various possibilities inherent in the viewpoint, 
or 'camera', of the interface of the RT3D MUVE, allowing a conscious experimentation 
with different modes of what is essentially a distributed avatar. In line with Flynn's ideas, 
the work rewards a reflective or contemplative navigation of the visual, spatial and sonic 
elements of the work, becoming familiar with audiovisual patterns that emerge from 
the coded virtual spaces that constitute the work; sound, vision and space all meld into 
a symbiotic compositional relation with the user such that they are indistinguishable 
as discrete media elements, evidencing the post-convergent nature of the work. By 
experimenting in this virtuosic manner, the user and the work enter a new relationship that 
is closer in spirit to the relationship between two performers than that of the relationship 
between artwork and user. In this way, Trace Aureity represents an investigation of non- 
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linear data flow in virtuosic artist/user interactive relationships, an audiovisual composition 
of relations along the lines of Deleuze's (1988: 123) 'composition of speeds and slownesses 
on a plane of immanence.' 




Trace Aureity enacts the modulation from data-as-data to data-as-display as its very work. 
In the state of data-as-data, the work does not exist except as an undifferentiated set of 
digital data. When this data is modulated into the state of data-as-display, it is modulated 
into patterns (colour, sound, shape, movement) and relationships (spatial, interactive) 
that may be retroactively narrativised by the user. Only then can it be said to exist. User 
engagement with the work facilitates this modulation, specific to moments in time, into 
data-as-display. There are both contingent selection, and autonomic engagements, within 
parameter frameworks, that produce specific colours, sounds, shapes, movements and 
interactions. These expressions are unique to that specific interaction and can never be 
exactly repeated, due to the autonomy given to the generated digital entities in modulating 
from data to display. 



Once such a set of relations as those between artist, work and user in Trace Aureity have 
been composed, new potentials for intimate or sensual engagement can be explored. This 
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is another of the intrinsic qualities of virtual art - its site of interaction can be very intimate. 
Removed from physical staging or curation, such as in an art gallery or a concert hall or 
a nightclub, the interactor is often interacting with a virtual work while they are physically 
alone or in a place that is familiar, a private space. This is the kind of space that perhaps 
parallels that suggested by Erik Davis' (1997) sensual reading of McLuhan's concept of 
'acoustic space.' 





Figure 7: Screenshot from Moaning Columns of Longing (Unsung Song 
#7) by Adam Nash, 2007-2009. Image and permissions provided by 

Adam Nash. 



Two other works of mine attempt to explore this relation in two different ways. First, The 
Moaning Columns of Longing (Nash, 2007) explicitly explores the nature of our emotional 
reaction to virtual experience. Staged in Second Life, the work enacts emotional and 
sensual blackmail. In doing so, it consciously exploits the user's understanding of the spatial 
and material analogies used to interface with the environment. This creates a technically 
mediated experience of emotion/affect that is aware of its own excess. When a user's avatar 
approaches the work, a virtual 'column' is spawned, glowing white and spurting glittering 
particles of joy while it declares its undying, faithful love for the user via the 'chat' facility 
built into the interface. Once the user leaves the space (that is, logs out), the column starts 
sending more and more emotionally manipulative emails begging the user to return. Every 
hour that the user does not return to 'touch' the column ('touch' is the Second Life interface 
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analogy used instead of 'mouse click'), the column becomes a little duller, a little shorter, 
and starts emitting a moaning sound that becomes louder as the hours pass. The hourly 
emails become more desperate and more emotionally demanding ("without you, I will die"). 
If the user does not return within 24 hours, the column will 'die', that is, in technical terms it 
will be deleted from the database permanently. If the user does return to 'touch' the column 
within 24 hours, the column returns to its full height, glowing brightly and rejoicing loudly 
over the chat channels, "[username] loves me! My existence has meaning!". Over the couple 
of years this work was in operation, some users maintained their columns for months at a 
time. During this time, these users returned to their column at least once every 24 hours, and 
received an email from their column every single hour for the entire time their column was 
'alive'. This nuanced composition of technologically mediated emotional exchange between 
the virtual and material calls into question the instrumentality of not only the technological 
interaction itself, but also the very means by which such an interaction can take place and 
the affective relationship between love and technology. It also amply illustrates the process 
of modulation from data-as-data to data-as-display and the complicity of the user in such a 
modulation. Until the user activates the work (both technically by logging in and conceptually 
by accepting the display parameters such as 'touch', 'visit', 'love,' etc), it is nothing more 
than a set of undifferentiated, plastic, digital data. Upon activation, the data-as-data is 
modulated, via a computational parameter framework, into data-as-display (sound, vision, 
words, emotions), the only mode by which it is really possible for a person to interact with 
the work. 




Figure 8: Screenshot from One, Another by Adam Nash, 2009. 
Image and permissions provided by Adam Nash. 
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The other work I want to discuss in this light, my work One, Another (Nash, 2009), also 
staged in Second Life, explored a combination of the concerns of both Trace Aureity and The 
Moaning Columns of Longing. Composed of a large flat platform of pink tiles, each tile chose 
its own saturation of pink based on its proximity to the centre, or heart, of the composition. 
Users could either wander around within the platform, triggering virtual entities programmed 
with very simple parameters, or simply wait for the virtual entities to emerge of their own 
"volition". These entities would themselves "wander around" constantly seeking to connect 
with other entities but always failing to do so since they were not programmed with the 
parameters that would allow them to understand such connections. Any movement, whether 
by a user's avatar or by a virtual entity, caused multi-sited ripples of affect throughout the 
work. Based upon Lacanian (Zizek, 2008: 66-67) concepts of lack and Julia Kristeva's (2002: 
178) subsequent sophisticated notion that 'imagination is a discourse of transference - of 
love,' feedback loops of interactions created complex audiovisual assemblages that would 
sometimes resonate for hours after the user had left. This work raises interesting questions 
about the causality and trace of affect, and the relationship between data modulation and 
affect. The display will not occur unless there is a user logged in, and yet the operations 
on the data continue. Once the user logs back into the environment, these operations are 
displayed as if they have actually occurred, and in this sense they can only retroactively 
be said to have occurred. Accordingly, the relationship between the modulation of data 
(between data-as-data and data-as-display) and the autonomy of affect are highlighted. 
Massumi says that "[a]ffect is the virtual as point of view" (emphasis in the original). It may 
be clearly seen in the artwork One, Another that the modulation from data-as-data to data- 
as-display is a kind of technical enactment of the virtual as point of view. At the same time, 
Massumi is right to caution against taking the visual metaphor too seriously, since affect 
- like data-as-data - is plastic and formless, until contingently specified. Specification (or 
modulation according to a parameter framework) may occur in or across any sensual mode, 
regardless of the ability to perceive of the participating bodies. (Massumi, 2002: 35) This is 
not to suggest a simple concordance between digital data and the virtual. Rather it is an 
attempt to identify the virtual aspects of a site where pre-individual affect meets interactive 
bodies that participate in both affect's capture and escape. 



Conclusion 



By working in the medium of digital data, these artworks raise numerous questions about 
the relation between the virtual and affect. Amongst them are the following: Is it the case, 
as Stephen Wolfram suggests, that this kind of complex generation of virtual/actual form 
through the interaction of very simple rules is constitutive of life itself? And, if so, what are 
the implications of this for the universality of affect? Can an immanently digital affect cycle 
be said to exist? (Chaitin, 1999: 108-109; Chaitin, 1998: 85) Brian Massumi (1995: 96) has 
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suggested that 'the measure of a living thing's potential interaction is its ability to transform 
the effects of one sensory mode into those of another'. If we accept this, connections are 
raised between the questions I have discussed in this paper about the nature of data-as-data 
and its modulation into data-as-display, along with the plexus of affective compositional 
relations that evolve in the convergence of virtual and material spheres. Can we therefore 
identify a relationship between what I have identified as the excess created by digitisation 
and the excess of affect as identified by Massumi and Guattari? (Bertelsen & Murphie, 2010: 
153) If so, then it would seem that the ostensibly different uses of the term 'virtual' - in 
its Deleuzian sense, in Massumi's affect-as-virtual sense and in the sense of the virtual as 
cyberspace or networked digital data - may in fact be closely related, and such a relationship 
still needs to be rigorously explored. Similarly, since the implications of biomediatic readings 
of affect theory can be actualised or experienced in the interaction between the material 
and virtual, what exploratory invitations does this open for artists working in virtual 
environments? 



These questions are current and pressing, and any answers are likely to be emergent and 
contingent. Part of the point of my ongoing virtual art practice is to attempt to propose 
questions that may be appropriate to our new situation. To this end, long-term collaborator 
John McCormick and I are currently working on Reproduction. (McCormick & Nash, 2010) 




Figure 9: Reproduction by John McCormick and Adam Nash, at Screen 
Space Gallery, Melbourne, Australia, 2011. Image and permissions 
provided by Adam Nash. 
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Reproduction is an evolving virtual environment of artificial audiovisual entities that 
exist in a symbiotic relationship with humans. In a physical gallery space, visitors spawn 
and manipulate entities via motion capture, while online users spawn and interact with 
entities through social networks - Facebook updates, tweets etc - and a custom web 
based interface. Because it's a symbiotic relationship, the artificial entities are not simply 
linear dependents on the actions of humans. Rather they evolve and grow according to 
the parameters of their ecosystem, which itself emerges from the agency of the entities. 
Many of these parameters are influenced by the (physical and virtual) actions of humans, 
and the artificial entities attempt to learn and anticipate human actions. Humans equally 
attempt to learn and anticipate the behaviors and reactions of the artificial entities. The 
result is a complex assemblage of human and virtual life that evolves and learns from its 
own participants. Virtuosic human users of the system (for example, dancers, musicians, 
gamers, or any person who is willing to spend the time) are able to engage in improvisatory 
performances with the artificial entities, where the artificial entities are making performative 
decisions of equal influence to human decisions within the performance, with human and 
digital entities responding to each other's actions. Casual visitors to the gallery are able to 
engage with the artificial life environment in a contemplative navigation based on movement 
in physical space. Online users of the work are able to engage in a more subtle and 
nuanced manner, entering into long- and short-term relationships with digital entities, who 
become 'friends' in both the social networking and material sense, influencing each other's 
behaviour. 



The work questions the nature of our virtual society by extending emerging social network 
practices. Is a 'friend' really a friend? What manner of my 'self is my online self? How much 
agency do these selves have and how do these multiple selves interact with each other and 
others? What is the nature of the affect facilitated in such a composition of relations? In this 
paper, I have attempted to show that virtual art raises new questions both about the nature 
of affect and relation, and about the ontology of digital data. Virtual art, I have suggested, 
enables us to more adequately reconfigure the concepts with which we approach data, in 
terms of its ontological status, because it foregrounds the key role played by modulation, 
and in the instability of the protocols that interpret data. I have also shown, through an 
exploration of my own art practice, that virtual art enables us to grasp the new kinds of 
affect opened up by the digital regime. 
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Biographical Note 



Adam Nash is internationally recognized as one of the most influential artists working 
in virtual environments. His work has been presented in galleries, festivals and online in 
Australia, Europe, Asia and the Americas, including peak festivals SIGGRAPH, ISEA, ZEROISJ 
and the Venice Biennale. He was the recipient of the inaugural Australia Council Second Life 
Artist in Residence grant. He was awarded an Ars Electronica FutureLab residency in 2009. 
He was awarded an Australia Council Connections Residency in 2009, for which he founded 
SquareTangle with colleague John McCormick. He founded the Australian Centre for Virtual 
Art with Christopher Dodds, which runs labs and workshops for existing and emerging artists 
to explore virtual media. He was shorlisted for the National Art Award in New Media at the 
Queensland Gallery of Modern Art in 2008. He has a PhD in Animation and Interactive Media 
from the School of Media and Communication at RMIT University, where he is a lecturer in 
Computer Games and Digital Art. 



Notes 



[1] Links and documentation concerning the artworks can be found at http://adamnash.net. 
au . 

[2] There must be an external protocol to facilitate modulation, protocols are intentional 
forces designed to facilitate modulation to a specific display state. Even were digital data to 
begin designing their own protocols they would still need an external, intentional, protocol 
that determines how they would do that. 
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Figure 1: 'Intimate Transactions Book Cover' (see Armstrong, 2006). 
Source: http://www.embodiedmedia.com/, with permission of Keith 



This article considers the 'co-affective' power (Ettinger, 2011: 13) of the new media artwork 
Intimate Transactions. Keith Armstrong (2005), artistic director of the Transmute Collective — the 
creators of Intimate Transactions — describes Intimate Transactions as collaborative, ecological, 
and concerned with relation. [1] In its most recent incarnation Intimate Transactions takes 
the form of a 'dual site networked installation' — 'two people' participate in the artwork from 
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'two different locations' (Armstrong, n.d.). In Sydney, where I encountered the work, these 
locations were the Performance Space in Redfern and Artspace in Woolloomooloo. [2] 



Participants engage with Intimate Transactions through active 'full body' movement. 
Through this, they engage with the animated 'creatures' in the 'virtual environment[s]' on a 
large screen (Armstrong, 2005; Hamilton and Lavery, 2006: 2). At times it is also possible to 
collaborate in a networked, 'moving together' with the other person (Massumi in Massumi 
and Zournazi, 2002: 223). This 'moving together' affects what occurs throughout the entire 
work. 



Ultimately, as Armstrong (2005) states, the aim of Intimate Transactions is a collaboration 
that will 'enrich' both the sensual and screen-environments. The work is not about individual 
or 'absolute control'. It is not only about 'me' and it is not about winning. Rather, in Intimate 
Transactions there is a complex, and 'intimate', bodily 'energy transfer' between participants. 
The design of the work encourages participants to move together, with the non-human 
creatures as well as the other person, in a 'co-creative' (Ettinger, 2006a: 122) collaboration. 
As such, Intimate Transactions as a whole is designed to operate, 'co-creatively' (Armstrong, 
2005), at a 'co-affective' level of experience (Ettinger, 2011: 13). In sum, the 'co-affective' 
activity of Intimate Transactions emerges from a larger 'trans-subjective' field (Ettinger, 
2006a: 111; Guattari, 1995: 6). This field involves the human participants as well as the non- 
human screen-creatures. 



Here I will explore the restorative powers of the collaborative and trans-subjective fields 
generated by Intimate Transactions (see also Armstrong 2006: 33). To do so, this article 
draws on the work of Bracha Ettinger, Felix Guattari, and Brian Massumi, amongst others. 
For these thinkers the 'trans-subjective level' of experience — precisely because it is 'co- 
affective' — holds 'ethical potentiality' (Ettinger, 2006a: 111 and 117). In general, there is a 
pressing need for explorations of the trans-subjective because such explorations 'may lead 
us to discover our part of shared responsibility in... events whose source is not "inside" the 
One-self (Ettinger, 1995a: 51). As such, the kind of exploration of the trans-subjective found 
in Intimate Transactions could be seen as providing a more embodied way of engaging with 
the contemporary 'eco-political' situation (Armstrong, 2006: 15). Intimate Transactions then 
is of particular interest because it is a work that aims to activate the 'ethical potentiality' of 
the trans-subjective more fully than many "interactive" works. [3] 



In this article I will first give a detailed description of the trans-subjective qualities of 
Intimate Transactions. As the title of the work suggests, Intimate Transactions is designed 
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to draw attention to, and filter action through, the trans-active (Armstrong 2005; 2006: 
25; Birringer, 2006). As such, Intimate Transactions deliberately challenges many standard 
notions of the interactive, and the activities and practices that result from these standard 
notions (see also Massumi, 2011: 39-86). 



The description of the work itself will be followed by a theoretical consideration of the 
'co-affective' nature of the artwork's 'transactivity' (Birringer, 2006: 109). In discussing 
this 'transactivity', Birringer writes that in Intimate Transactions '[t]he site of the body is 
a transactional collectivity; fluid, transitory, ungrounded' (Birringer, 2006: 109). [4] Taking 
on board Massumi's call for a rethinking of 'interactivity' in affective and 'relational terms' 
(Massumi, 2011: 52 and 67; see also Fritsch, 2011 and Brunner and Fritsch, 2011), the article 
considers 'transactivity' from the complementary perspectives of relationality, the trans- 
subjective, affect and ethics. 



As I have already begun to suggest, this 'transactivity' (Birringer, 2006: 109) is different 
to more conventional notions of the interactive. It is different because, with a focus on 
the trans-, we move away from thinking "interaction" only as occurring between 'already- 
constituted' subjects (and objects). In moving toward the trans-subjective, we move to 'the 
primacy of affect, relation and of the in-between (Massumi, 2002: 24; 1997a: 175; 2000a 
and 2011: 39-86). 



Following the work of Massumi in particular this article argues for the importance of a 
'tending' to this affective level of experience, both in designing "interactive" art — such 
as Intimate Transactions — and in life more generally (Massumi, 2000a: 216; 1997a; 
2011; see also Ednie-Brown, 2007). For Pia Ednie-Brown (one of the collaborators on 
Intimate Transactions) the kind of "design ethics" involved here must embrace a 'striving 
for a balance between affecting and being affected' (Ednie-Brown, 2007: 329). Intimate 
Transactions also makes it clear that to remain ethical and sustainable such tending 
to the affective level of experience must foster 'diversity' in 'collaboration' and avoid a 
consumption of difference (Ednie-Brown, 2007: 323; see also Ednie-Brown and Mewburn, 
2006). 
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Trans-subjective Collaboration and a 'Logic of Affect 




Figure 2: 'Participant navigating'. Image by David McLeod. 
Source: http://www.embodiedmedia.com/, with permission of 
Keith Armstrong 



In Intimate Transactions two people in different physical locations engage with a large 
'screen-space' (Armstrong 2006: 26). In order to engage with the work the participants stand, 
tilted slightly back on (what the artwork's creators call) 'identical Bodyshelves'. [5] Lisa 
O'Neill explains that the shelf is comfortable but that its backward tilt puts the participants 
in a 'slightly unusual position' (O'Neill, 2006: 38). From this tilted position the participants 
move their entire body, rolling the back and shoulders against the Bodyshelf in order to 
navigate the world on the screen. They also shift the weight of their bodies on the mobile 
platform on which they stand (see O'Neill, 2006). Engagement with the world on the screen 
involves a strange 'dance', more or less on the spot, from which the body-shelf/platform 
picks up bodily movement (Ednie-Brown, 2007: 244; see also O'Neill, 2006). 



The movement on the shelf enables participants to engage with the work and its screen- 
spaces — the worlds of the non-human creatures (Armstrong, 2006: 26). The movements 
of the body on the shelf also 'activate' the participants' avatars, which can enter what 
becomes at times a shared realm in the 'virtual world' (shared simultaneously across the two 
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screens in the different spaces) (O'Neill: 2006: 36-38). In this 'shared space' the avatars of 
the two participants can 'meet' and move together as one semi-merged avatar (Armstrong, 
2006: 27-28; O'Neill, 2006: 41). Through all this, as Ji II ion Hamilton explains, the '"Bodyshelf" 
requires whole body movement to activate the motion sensors embedded in the surface. This 
shifts the participant from a relatively passive wrist/hand interaction with the interface to a 
physically active, whole body engagement with it' (Hamilton, 2006: 120). 



However, the Bodyshelf does not only pick up and transfer the movements of the body on 
the shelf onto the avatars on the screen. The shelf also plays an active part in the 'immersive 
sound-scape', as the bodily 'motion in space generate[s] the feedback of the sound' 
(Webster, 2006: 60 and 67). This is an often intense and unusual sound. 



Finally, the Bodyshelf transmits vibrations, based on the other person's movements, onto 
the lower back of the body. The vibrating devices in the Body-shelf, 'are activated during 
the "meeting" of the two. ..avatars, when they become locked together in joint movement' 
(Ednie-Brown and Mewburn, 2006: 80-81). 




Figure 3: the rubber device. Video still. Source: http:// 
www.embodiedmedia.com/, with permission of Keith 
Armstrong 



Another aspect of the vibrating quality of the work emerges via a rubbery 'garment' (with 
a pink border) that is looped around the neck, like a 'pendant', and strapped loosely to the 
abdomen. Inside this garment there is a device that transmits vibrations onto the stomach of 
the participants. In this case the vibrations are based on the engagement with the creatures 
in the 'screen-world' (Ednie-Brown and Mewburn, 2006: 80-81; Armstrong, 2005). These 
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vibrations on the stomach, together with the vibrations on the lower back, constitute the 
most intimate qualities of the work. With these vibrations one literally feels the movement of 
others (see also Hamilton, 2006: 124). Pia Ednie-Brown and Inger Mewburn worked with the 
Transmute Collective in designing these more intimate vibratory components of the work. 



The intensity of the sound enhances the experience of intimacy. Guy Webster (sound director 
of the Transmute Collective) explains that '[t]he vibrations in the Bodyshelf, and in the 
pendant. ..are all controlled by sound'. For Webster this is an important aspect of the 'sound 
design' because the 'sounds can actually reach out and touch you' (Webster, 2006: 62). 



Body, eyes, animated images and creatures, intense sounds and the bodily vibrations are 
brought into a "co-territorialised" and networked milieu. A partial and distributed subjectivity 
emerges with this. Armstrong (2005) explains that for both participants the experience is 
one of transversally engaging with, and feeling the movement of, an excessive if unknown, 
anonymous body (one does not see the other participant before or during the "interaction"). 
In fact, it is not initially clear whose or even what movement we experience in the vibrations 
emerging from the 'haptic devices' in the Bodyshelf, the 'immersive sound' (produced by 
bodily movement) and the vibrations felt on the stomach (Armstrong, 2006: 7). Yet, over 
time, a strong sense of affective 'co-creation' emerges, between the human participants, the 
non-organic aspects of the work (including the technology), and the non-human creatures on 
the screens (these screen creatures are described by the artists as 'Force of Change', 'Force 
of Permanence', 'Force of Instability', 'Force of Conflict', 'Force of Torment' [Lawson and 
Foley, 2006: 56-58]). 



At the same time, it is important to note that in this 'co-creation' the work does not collapse 
into undifferentiation (see Ednie-Brown and Mewburn, 2006). Even though the experience is 
indeed 'co-affective', the felt differences between the participants and the screen creatures 
remain, even as transactions occur. In other words, differences are expressed even during 
the meeting of bodies and creatures. Rather than mergers, there are encounters via the 
felt vibrations, the visuals and the sound (Armstrong, 2006: 33 and Webster, 2006: 69). 
I will shortly discuss the precise moment at which these encounters take an ethical turn 
(encouraging 'relational difference in co-emergence' [Ettinger, 2006b: 72]). In order to 
discuss this ethical turn, it is first necessary to gain a better understanding of the various 
environments on the screens. 
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Figure 4: 'Creatures and Sample Internal Images'. 
Source: http://www.embodiedmedia.com/, with permission of 
Keith Armstrong 

In Intimate Transactions, there is a complex layering of different screen-worlds (Armstrong, 
2006: 27; Hamilton, 2006: 116). At times, participants, with their own avatars, can engage 
with the more "local" screen environments of-the non-human creatures (Armstrong, 2005). 
As mentioned, the various forces of the creatures are expressed in part as vibrations, felt on 
the stomach when participants engage with them (Ednie-Brown and Mewburn, 2006: 80-81; 
Webster, 2006: 69). However, even in this phase of the work, the actions of avatars change 
more than the environments of the creatures. They have effects throughout the system (and 
each person's avatar is presented as a 'shadow' avatar in the other participant's screen- 
space [Armstrong, 2005; 2006: 27]). At other times, there is a direct participation in a 'shared 
space' (Armstrong, 2006: 27-33). Here the avatars reflect the movement of both participants 
within a shared screen-space. It is in this environment that the separate avatars can meet 
and move together. [6] 



Armstrong (2005) emphasises the importance of the different screen environments. He 
explains that at one stage of the work it is possible for the individual participants to 
impoverish the world of the non-human creatures. In these more 'local' spaces the human 
participants can, individually, take away 'objects from' the non-human creatures in order to 
take possession of them and 'incorporate. ..these objects into their own avatars' (Armstrong, 
2006: 27). At the beginning of the experience, the participants are told the following: 



fibreculturejournal.org FCJ-149 37 



FCJ-149 Affect and Care in Intimate Transactions 



You can take things away from your Creatures, but in order to return these you must 
interact with the other person. 

How you treat these Creatures will ultimately affect what you see, hear and feel and 
what the other person sees, hears and feels ... (cited in Armstrong, 2005). 

Armstrong points out that it is possible to not 'work collaboratively with the other person'. 
However, the less one does so, and the more one takes away from the environments of the 
non-human creatures, the more impoverished the 'immersive' world becomes. 
The experience becomes tame. It loses its intensity. This loss of intensity is 'indicated by 
a rapidly increasing, overall sluggishness, lessening brightness and inability to transact 
smoothly' (Armstrong, 2005). Webster explains that '[a]s the effect moves across the whole 
spectrum, all the imagery starts to become lethargic and that's directly represented in the 
sound' as well (Webster, 2006: 66). It is clear then that non-collaboration is not encouraged by 
the very design of the work. 




Figure 5: 'Force of Change - Internal Composition'lmage by Benedict Foley. 
Source: http://www.embodiedmedia.com/, with permission of Keith Armstrong 
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In order to re-"enrich" all the screen-spaces, including the worlds of the non-human 
creatures, and indeed the gallery spaces, the two human participants have to work 
together. For one thing, their avatars have to join. They must meet in a 'trans-subjective' 
movement based on what Armstrong (2005) terms transversal, 'networked and cross- 
affective processes'. These affective processes are 'multidirectional' (Ettinger, 2006b: 64) 
and, as mentioned, take place in a shared space (Armstrong, 2006: 27). In this shared 
space participants can work together and move together to heal the creatures and 'restore' 
environments that may have suffered over-'consumption' (Armstrong, 2006: 29, 27). 




) 

f f 



Figure 6: 'Person to Person Interaction Screen'. Image by Keith Armstrong. 
Source: http://www.embodiedmedia.com/, with permission of Keith Armstrong 



Ecological responsibility here becomes distributed (networked). It emerges from an affective 
(trans-subjective) field shared across the two gallery spaces. Together, the vibrations, the 
'immersive sound' and the whirling in the screen-worlds make the movement of this shared 
trans-subjective affect felt. Affect here clearly emerges from the activation of 'our (collective) 
movements' (Birringer, 2006: 112). As such, it is in the activation of this collective sphere 
that Intimate Transactions takes on an ethical and restorative turn (see Armstrong, 2005). 
One aspect of this is that the environments of the non-human creatures can only be re- 
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energised if the participants actively surrender control, not that they ever have 'absolute 
control' (Armstrong, 2005). Surrendering control, they can meet in a trans-subjective 
movement that affects and restores the energy of the entire work. In sum, the participants 
have to 'cooperate to restore the creatures and the energy of their own/shared environment' 
(Armstrong, 2006: 29). As noted, Armstrong (2005) refers to this cooperative working process 
as 'cross-affective'. 



Here we can build on Armstrong's idea that much of the "action" in Intimate Transactions 
takes place on a 'cross-affective' level of experience. I am taking this idea of 'cross-affective' 
"action" a little further, seeing the potential 'co-creative' activity (Ettinger, 2006a: 122; 
Armstrong, 2005) involved as not only 'cross-affective' but also 'co-affective' (Ettinger, 2011: 

13). That is, affect not only crosses between various actions, but arises from within, and even 
perhaps as, the 'transactivity' involved (Birringer, 2006). This is at the distributed heart of 
what can be called a 'logic of affects'. Part of the achievement of Intimate Transactions is 
that it so emphatically emphasises an entire 'logic of affects rather than a logic of delimited 
sets' (Guattari, 1995: 9) [7]. In the latter, the 'logic of delimited sets', a collection of 'discrete 
elements' (such as participants and technical elements) come first and last, with something 
like "interaction" occurring, only secondarily, between them (Massumi, 2000a: 191). 
However, a 'logic of affects' is concerned with trans-subjective, 'pre-personal' and 'collective' 
fields of experience. A 'logic of affects' is 'polyphonic' (Guattari, 1995: 9 and 1) 'multi-polar', 
(Guattari, 1996: 158) as well as 'co-creative'. [8] It is precisely because the 'logic of affects' 
is 'multi-polar' and 'co-creative' that it can be understood to involve 'transactivity' across 
emergent subjectivities, rather than interaction between 'already-constituted' subjects or 
objects ('delimited sets') (see Massumi, 1997a: 175; 2011: 39-86). It is the rigorous attempt 
to design for engagement within the 'logic of affects' that makes Intimate Transactions 
a matter of 'transactivity', more than interactivity (the latter of which can sometimes 
seem more concerned with the 'logic of delimited sets'). With its focus on 'co-affective' 
'transactivity', Intimate Transactions thus appears to challenge the more conventional 
notions of interaction. 



Prior to writing about Intimate Transactions and 'transactivity' Birringer proposed ways of 
categorizing 'various types of interactive environments (sensory, immersive, networked and 
derived environments)'. He explains that '[w]hen the parameters of these are mixed, we 
speak of mixed reality or hybrid environments'. However, according to Birringer Intimate 
Transactions is different because it involves 'transactivity': 
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Intimate Transactions is another category, a transactive environment, involving 
telematic performance with distributed action, where images and sounds are created 
not simply to be transmitted from one location to another, but to cooperate in an 
evolving feedback loop via a virtual ecology. (Birringer, 2006: 108-109) 

As fits 'a logic of affects', the primary concerns of Intimate Transactions are not only 
relational then but also 'ecological' (see Armstrong, 2005 and 2006). They are ecological in 
two senses. First there is a focus on ecology understood in terms of the complex dynamics 
of relations in a given situation, with an emphasis on changing the way that these dynamics 
are approached/experienced. Second, the design of Intimate Transactions fosters the 
possibility that experiencing the dynamics of this ecological relationality, in this case very 
intimately within an artwork, may change approaches to more general environmental issues. 
Armstrong argues that 'the way we approach design can have an enormous impact upon 
the way that we interact with the world. It can potentially change the way that we approach, 
and therefore understand, ecology' (Armstrong, 2006: 15). [9] 



'[E]cological concerns' then, form a key part of the "transactive design" of Intimate 
Transactions (Armstrong, 2006: 13). We have seen that in order to produce more sustainable 
changes within the work, the "action" in Intimate Transactions must not only emerge from 
isolated, already individuated bodies or selves (see Armstrong, 2006: 13-16) (from what I 
referred to above as 'the logic of already 'delimited sets'). If engagement with the work 
stops there, this leads to a kind of 'ecological crisis' in what become diminished 'audio- 
visual' and 'tactile' worlds (Armstrong, 2005). Instead of stopping at individual action, 
in order to care for the virtual environments, participants in Intimate Transactions are 
encouraged to engage fully in the 'relational potential' (Massumi, 2000a: 202) of the 'logic 
of affects' (the collective, 'co-creative' and 'trans-subjective levels' of experience). Only 
if "action" — in this case movement — becomes distributed across a larger affective field 
engaging both participants (as well as the screen creatures) in a 'co-affective' collaboration 
does the work take an ethical and reparative turn toward a restoring of ecological balance. 
From this collaboration a more rewarding experience of the work itself also emerges (see 
Armstrong, 2005). 



The focus on 'co-affective' collaboration in Intimate Transactions reflects a less individualised 
notion of affect, shared by the thinkers discussed in this article. Erin Manning, for example, 
notes that 'affect does not rest in the individual'. Rather, affect is 'collective' and of the in- 
between (Manning, 2010: 117 and 122). [10] Guattari writes about the "sticky" 'transitivist 
character of affect' operating within a 'multi-polar affective composition' (Guattari, 1996: 
158). Massumi considers affect 'transsituational' (Massumi, 2000a: 185) and Ettinger thinks 
affect in terms of 'co-affectivity' (Ettinger, 2000: 98). For all these thinkers affect is located, 
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beyond the emotional states of the human individual, in a larger distributed field. For these 
thinkers and, as taken up in the design of Intimate Transactions, 'the logic of affects' is 
generative, collective and 'distributed' (Hamilton, 2006: 118; Armstrong, 2005; see also 
Ednie-Brown, 2007: 178). [11] 



These approaches to affect are shared by Ednie-Brown (2007). They are present in the design 
philosophy and architecturally inflected art practice she brought to the collaborative creation 
of the 'haptic components' of Intimate Transactions (Armstrong, n.d.). As mentioned, Ednie- 
Brown has developed a "relational design ethics" 'striving for a balance between affecting 
and being affected' (Ednie-Brown, 2007: 329). 



Belonging 



We have seen that, as an ecological work, explicitly concerned with sustainability (Birringer, 
2006: 108), Intimate Transactions aims to create a care for the world that tends to a more 
distributed sense of 'ethical responsibility' — in, across, and beyond the artwork (Armstrong, 
2006: 15). For Armstrong, Intimate Transactions thus works at the level of the 'eco-political' 
(Armstrong, 2006: 15). Massumi articulates the general aim of this kind of political concern 
very well, stating that '[t]he "object" of political ecology is the coming-together or belonging- 
together of processually unique and divergent forms of life' (Massumi, 2000a: 216). 



For Massumi 'there are ways of acting upon the level of belonging itself, on the moving 
together and coming together of bodies per se' (Massumi in Massumi and Zournazi, 
2002: 223). Massumi also suggests that '[e]thics is a tending of coming-together, a caring 
for belonging as such' (Massumi, 2000a: 216). In general terms, 'caring for belonging' 
is relational yet this relationality is 'of the middle', that is, it does not link the 'already- 
constituted' (Massumi, 1997a: 175). Rather, for Massumi as for Gilbert Simondon 'a true 
relation is that which constitutes the terms that it connects' (Flanders in Simondon, 2009: 
15, see also Massumi, 1997a). The relation itself is a 'co-creative' process, which produces 
new individuations (see also Brunner and Fritsch, 2011). I have outlined above how Intimate 
Transactions encourages relation - the work is 'highly relational' (Armstrong, n.d). Yet for the 
work to be truly transformative this must involve a co-constitutive 'transactivity' productive 
of new individuations and milieus. 
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Of course, activating the more affective level of experience does not automatically involve 
a care for 'belonging-together'. Intimate Transactions is affective throughout, yet subtle in 
its foregrounding of different kinds of affective engagement, some more directed towards 
'belonging-together' than others. If participants only work individually and collect object 
from the non-human creatures there is no care for 'belonging-together'. As we have seen, 
it is in the more collaborative phases that Intimate Transactions explicitly encourages 
individuations that involve 'a caring for belonging as such' (Massumi, 2000a: 216). 



This care 'for belonging' involves a "micropolitical" approach to ethics in that it works at 
the level of our bodily habits (see Massumi in Massumi and McKim, 2009 and Guattari 
in Guattari and Rolnik, 2008). In this regard Intimate Transactions attempts to resist, by 
deterriton'alising, the more destructive habits of the human body. These destructive habits 
come to life (and it is hoped that participants become more aware of them) at the moments 
in Intimate Transactions at which it is possible for the participants to impoverish the world 
of the non-human creatures. Intimate Transactions challenges this impoverishment with 
the offer of a deterritorialisation that can 'restore' and re-enrich the 'virtual world' (O'Neill. 
2006: 41) in a 'moving together'. In this restorative movement the participants at different 
locations move together and participate in a collaborative individuation of new subjective 
possibilities and worlds. In general, it is hoped (but never guaranteed) that these new 
individuations will avoid a 'conservative' or exploitative 'reterritorialisation of subjectivity' 
(Guattari, 1995: 3) and produce a 'caring for belonging' (Massumi, 2000a: 216). To reiterate, 
such a care does not emerge from the interaction between 'already-constituted' bodies or 
selves but from the midst of a distributed, vibrating and trans-subjective affect. The care 
does not emerge from autonomous subjects but from relational 'not one-ness' (Ettinger, 
1992: 178, see also Hamilton, 2006). 



For thinkers such as Ettinger, Simondon or Erin Manning the question of individuation is 
complex. Individuation and the body are 'always more than one' (Manning, 2010: 117). It 
is this 'not one-ness' that Intimate Transactions taps into. Yet, again, this 'not one-ness' 
involves no simple unity, not even in a 'coming-together'. Rather it is a question, as in 
Intimate Transactions, of activating what Pia Ednie-Brown and Inger Mewburn (the creators 
of the haptic components) call the 'undeniable difference between us'. They write about the 
haptic components as addressing 'a power that vibrates with the texture of difference'. 



In our opinion, the value of this particular project lies in actively exploring ways and 
means through which we might deal with that difficult political, social, ethical and 
perhaps universal problem we keep repeating: the difficulty of forming a sense of 
shared experience amidst the undeniable difference between us. (Ednie-Brown and 
Mewburn 2006: 87) 
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So the 'not one-ness' of Intimate Transactions does not only concern the simple fact that 
there is more than one participant. It also concerns the more complex reality of participation. 
Participation is never foundationally about "the individual" or a "becoming one", but rather 
about a collaborative becoming that emerges from ongoing individuation — a 'relational 
movement' in Manning's terms (2009: 29). The main loci for this are the trans-subjective 
vibrations of bodies and existential territories as they 'come... into being' (Massumi, 2000a: 
201). 



The work of Simondon is again illuminating. Venn explains that for Simondon 'Being... 
is neither pure unity nor pure plurality' (Venn, 2010: 150). To think about this in terms 
of the 'production of subjectivity' (Guattari, 1995: 1) it could be said that the 'subject 
does not coincide with the individual'. Rather, in what has been referred to here as the 
"trans-subjective", '[t]he subject in the Simondonean problematic is an ensemble of pre- 
individuated and individuated realities, thus pregnant with "virtualities" or "potentials"... 
and still open to further individuation through the collective or group' (Venn, 2010: 150). It 
is this potential for further individuation that 'comes into being' (Massumi, 2000a: 201) in 
the 'transindividual' (Simondon, 2009: 8) group processes and experiences in the shared 
screen-world of Intimate Transactions. Throughout there is the question of relations between 
previous individuations and new individuations. 



As mentioned previously, Ettinger and Guattari, like Simondon, place great value on 
transindividual experience in the individuation of subjectivity. Ettinger develops this with her 
concepts of the matrixial and metramorphosis and Guattari with the notion of transversality. 
Both thinkers can further our understanding of the ethics involved in the affective 
'transactivity' of Intimate Transactions. Ettinger first. 



Metramorphic Transactivity 



Ettinger's concepts of the matrixial and metramorphosis are intended to account for the 
potential ethical and generative dimensions of 'trans-subjective' encounters (Ettinger, 2006a: 
111 and 117). They are thus useful when considering what in this article, following Birringer, 
has been referred to as 'transactivity'. 



Trans-subjective encounters include 'aesthetic encounters' with(in) art, a major concern 
for Ettinger (Pollock, 2004: 7; Ettinger, 2006b). Ettinger's concepts describe a relational 
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distribution of subjectivity (across bodies and aesthetic 'elements' for example) (Ettinger, 
1993a: 39; Pollock, 2004: 7). As such, her concepts enable us to think an artwork's active 
participation in the ongoing individuation of subjectivity (Ettinger, 1997: 639). [12] In short, 
'the matrixial stratum of subjectivization reveals subjectivity as an encounter of co-emerging 
elements through metramorphosis' (Ettinger, 1996:125). 



Let us begin with the matrixial. What Ettinger (2007) terms the matrixial is situated in a 
larger 'trans-subjective sphere'. Ettinger thinks of the matrixial (the 'trans-subjective') as a 
feminine force. However, this does not imply that the matrixial is 'about women' as such 
(Ettinger, 1993b: 18). Ettinger comments that the matrixial is not about 'Oedipal gender 
difference' (2006b: 69). Rather, the matrixial concerns the 'trans-subjective' ("the feminine" 
if you like) for all genders (see also Massumi, 2000b: 31). Ettinger refers to this 'trans- 
subjectivity' (1999: 15) as an 'enlarged subjectivity' or 'subjectivity-as-an-encounter' (1996: 
133 and 145). The main point is that the matrixial conceptualises a distributed moment in 
the production of subjectivity (Ettinger, 1996: 153) - 'trans-subjectivity in severalty', to again 
use Ettinger's conceptual vocabulary (1999: 15). In Intimate Transactions, the distribution 
involved moves beyond the individual body or psyche, into what Ettinger (2006b) has termed 
a larger matrixial borderspace. This 'borderspace' has its own force - the force of what 
Ettinger calls 'relational difference in co-emergence' (Ettinger, 1995b: 30). I am suggesting 
here that the collaborative phases of Intimate Transactions allow participants to begin to 
experience the trans-subjective and ethical force of this 'borderspace' more directly. 



In Ettinger's (2009) terms Intimate Transactions could be described as an 'encounter-event'. 
This kind of event involves encounter between what Ettinger terms 'co-emerging l(s) and 
non-l(s)' (Ettinger, 1996: 127). This is a productive co-emergence between that which is 
emerging as "me" and that which is not "me", but which co-emerges with "me". 'Non-l(s)' 
may include 'inanimate objects' (Ettinger, 2006b: 90) or 'non-human' life (Pollock, 2004: 
7) — such as the creatures of the screen-worlds of Intimate Transactions. Crucially, both 'l(s) 
and non-l(s)' are 'partial'. They are part- 'elements' or 'partial-subjects' within the broader 
and ongoing production of subjectivity (Ettinger, 1996: 154 and 129). In addition, like the two 
participants in Intimate Transactions, 'l(s) and non-l(s)' may be 'unknown' to each other — 
even, as in Intimate Transactions, never really meet, remaining anonymous (Ettinger, 1997: 
638). They can nevertheless still connect in some way and affect each other (see Ettinger, 
2006b). Ettinger explains that 'the trans-subjective level, as the time-space of encounter- 
event is shared by several intimate-anonymous l(s) and non-l(s)' (Ettinger, 2006a: 111). 



Ettinger's conceptualisation of a quality of subjectivity in terms of 'co-emerging I and non-1' 
(Ettinger, 1996: 134) reminds us of Armstrong's commitment to 'ecological subjectivity'. 
Armstrong explains: '[w]ith my interest in ecological subjectivity I was exploring ideas 
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of things that are close to what I understand as "me" and then moving towards things 
that appear to be "separate from" or "unknown to me", yet that I understand my body 
is undivided from' (Armstrong in Armstrong and Gallash, 2004). I would argue that this 
'ecological subjectivity' works at the level of the 'co-affective': in a sense it is matrixial. 



Ettinger writes about the matrixial in terms of 'co-affectivity' (Ettinger, 2000: 98). She 
writes of a 'co-poietic transformational potentiality' that moves beyond 'inter-subjective 
relationships' and 'verbal communication' — and beyond "interactions" between clearly 
identifiable bodies, objects or subjects (Ettinger, 2005b: 703). 



It is important to note that in the matrixial realm trans-subjective expression becomes 
not only 'co-affective' but possibly also ethical (see Ettinger, 2009). It is such 'co-affective' 
and ethical expression that may emerge in Intimate Transactions. Without ever gaining 
'absolute control', the actions of the participants affect the entire work and all the screen- 
worlds (Armstrong, 2005). As mentioned above, this becomes particularly evident when, 
in the collaborative phases of the work, the participants can move together. At these 
moments there is 'not a filtering of the other through the one' (Ettinger, 1993a: 68). Rather, 
what is experienced is what Ettinger terms 'metramorphosis'. [13] 



Metramorphosis, in the individuation of subjectivity, and in art, is concerned with 
transformative engagement and 'co-affective' becoming. It is different to metamorphosis 
in that metramorphosis is without a resolution of these becomings into a 'unity'. One 
"thing" does not simply become "something else" (Huhn, 1993: 8). Thus Ednie-Brown and 
Mewburn's focus on 'the undeniable difference between us' in their design of the vibrating 
components for Intimate Transactions (Ednie-Brown and Mewburn, 2006: 87). For Ettinger, 
metramorphosis concerns the between of this 'undeniable difference'. 



'Metramorphosis is the becoming threshold of borderlines. Through such borderlines, an 
ever continuing negotiation between l(s) and non l(s) passes'. Ettinger writes that '[w]hen 
changes occur in the borderline between two fields, they produce changes in both fields' 
(Ettinger, 1993b: 13). 



Rosi Huhn explains that 'in contrast to metamorphosis, each of the new forms and shapes 
of metramorphosis does not send the nature of each of the preceding ones into oblivion or 
even eliminate it'. Unlike metamorphosis, metramorphosis 'leads an existence of multitude 
rather than unity. It is in this quality that the program of the "Matrixial" conception of the 
world manifests itself (Hunh, 1993: 8). 
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In sum, metramorphic processes are different to metamorphosis. One side of the process is 
not left behind for the sake of the becoming of the other. Both are transformed. "Interaction" 
here truly begins to take shape as 'transactivity'. Arguably in being designed for ecological 
trans-activity, Intimate Transactions also moves toward the metramorphic. 



As we have seen, because metramorphosis is 'co-affective', it provides a possibility for 
'ethical encounter' (Ettinger, 2006a: 132). This implies that creating, collaborating, thinking, 
designing or making artworks metramorphically also provide a possibility for 'ethical 
encounter'. Ettinger's ethical approach to encounter takes account of the inherent and non- 
reductive difference in any 'encounter event' (Ettinger, 2009). It allows for the always present 
'difference' in the 'co-emergence'. 



It is this kind of non-reductive difference that is experienced in Intimate Transactions. As 
we have seen, the design of the work enables participants to sense the productive and 
'undeniable difference between us' (Ednie-Brown and Mewburn, 2006: 87). This is one 
of the qualities that makes this complex work so powerful. Beyond the 'visible changes' 
(Webster, 2006: 69) you feel and hear the 'undeniable difference' of the creatures and the 
'co-emerging' other participant. 



As I have mentioned, the encounters with the creatures are felt through vibrations in the 
stomach 'garment' (the rubbery device). These sensations differ depending on which creature 
is encountered (Ednie-Brown and Mewburn, 2006: 81-82). Webster has described the specific 
sensations in the various encounters with the different creatures in some detail. He explains: 



Permanence is based on the idea of a rock, and it responds minimally to your 
presence. The sounds for Permanence are based on a deep, meditative series of 
sounds. There are minimal sound and motion changes, but Permanence responds by 
vibrations 

Torment, on the other hand, is based on the personality of a voracious insect. And it 
doesn't like your presence. It moves a lot. So it is very difficult to engage with. The 
sounds of Torment are breathy. Rhythmically, it is very fast and shaky... 

Each creature has its own designated series of vibrations that change as you interact 
with them. For instance, Conflict starts pretty harshly and it becomes harsher, until it 



fibreculturejournal.org FCJ-149 47 



FCJ-149 Affect and Care in Intimate Transactions 



starts to shake you. Since you're right in the middle of the space, your whole world 
becomes intense. (Webster, 2006: 69) 

However, not only do the participants feel the specific differences of the various creatures. 
Recall that when participants are 'moving together' to 'restore' the 'virtual world[s]' they also 
feel each other. O'Neill explains that 'the movements of the other participant are relayed by 
pushes and pulls that are felt through the backboard of the shelf (O'Neill 2006: 41-42). It is 
actually possible to 'feel' the 'direction' in which the other person is moving (Webster, 2006: 
66): possible to 'feel the directional push of the other' (Ednie-Brown and Mewburn, 2006: 81). 



To employ Ettinger's vocabulary then: in Intimate Transactions we feel the movement of 
'non-l(s)' and the exchange between 'l(s) and non-l(s)'. We feel the vibration or pull of the 
other in a 'moving. ..and coming together of bodies' as this exchange (Massumi in Massumi 
and Zournazi, 2002: 223). Creatures, Bodies, Bodyshelves, sound, the haptic device on 
the abdomen, and the 'screen-worlds' become sites for this exchange as each participant 
is in affective encounter with their own gallery space, the localised screen-worlds and 
their creatures, the 'shared world' and the other participant (Birringer, 2006: 107). It is 
here we understand the different (perhaps more ethical) approach to interactive art and 
design suggested both by aspects of Intimate Transactions and Ettinger's concepts of the 
matrixial and metramorphosis. For, as interaction becomes 'transactivity' we have to enter 
into a 'co-affective' movement - metramorphosis - in order to 'restore' worlds (in Intimate 
Transactions, the worlds of the screen creatures), to share worlds, or just to bring back some 
energy and joy to encounters within and between the separate gallery spaces (see Birringer, 
2006). 



Metramorphic 'becoming-together' then (Ettinger, 1995a: 30), is a process that gives 
'expression' to 'differential mutual emergence' (Massumi, 1997b: 779). Furthermore, as I 
have detailed above, metramorphic 'becoming-together' is important in that it provides the 
'logic of affects' with ethical force. Again, it is such 'becoming-together', with difference at 
the core, that begins to be rendered perceptible (Deleuze and Guattari, 1987: 281) in Intimate 
Transactions. Intimate Transactions thus encourages the matrixial quality of subjectivity — 
'relational difference in co-emergence' (Ettinger, 2006b: 72) — to become the main aspect 
of the experience for participants. This gives the experience an ethical pull. Recall that the 
haptic quality of the work in particular enables one to feel the force of this. Ednie-Brown 
and Mewburn writes that 'its fruits lay in understanding something of the nature of the 
power situated between us'. As mentioned this is 'a power that vibrates with the texture of 
difference' (Ednie-Brown and Mewburn, 2006: 87). For me it is also this slightly surprising 
quality of the work — surprising because the vibrations on the stomach and lower back are so 
intimate — that enables one most effectively to enter into a creative and restorative becoming 
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with the creatures and the other participant (O'Neill: 2006: 41). It is this kind of becoming — 
not always easy or harmonious — that can tend to the complexity involved in caring for the 
contemporary world and a differentiated 'belonging-together' (Massumi, 2000a: 216). 



Transversal Group Eros 



This 'belonging-together' is also central to Guattari's work. Here we can extend the 
discussion of metramorphic 'transactivity' toward collective life as, for Guattari, 'the 
production of subjectivity' (Guattari, 1995: 1) needs to be thought in relation to the group - 
as 'group subjectivity' (Tinnell, 2011). Even the supposed "individual" is a 'group subjectivity'. 
This group subjectivity involves a consideration of 'existential territory' understood here 
via what Guattari calls transversality (see also Armstrong, 2005 and 2006 and Genoski, 
2000). Guattari writes that 'it appears opportune to forge a more transversalist conception 
of subjectivity, one which would permit us to understand both its idiosyncratic territorial 
couplings (Existential Territories) and its opening onto value systems (Incorporeal Universes) 
with their social and cultural implications' (Guattari, 1995: 4). 



Guattari's 'transversalist' notion of subjectivity, together with the idea of existential 
territory, enable Guattari to develop a theory (and a politics) of subjectivity with multiplicity, 
collectivity and difference at the centre (see Guattari 1995 and 1996). Thus Guattari develops 
'a conception of the individual as fundamentally a group, a social subject, a group subject' 
(Genosko, 2000: 156). Gary Genosko explains that 'Guattari stakes a sociological claim with 
Eros, while Freud [in the end] chooses an anti-sociological principle in the name of Thanatos 
[the death or destructive drive]. For Guattari, Eros and the Group triumph over Thanatos and 
the individual': This implies that Guattari enforces 'Eros and Society over the death drive and 
the narcissistic individual' (Genosko, 2000: 155). [14] 



This is interesting in terms of the experience of Intimate Transactions. Intimate Transactions 
makes us very aware of our more destructive habits and drives, especially in the phase of 
the "interaction" during which we can collect objects from the non-human creatures. At this 
stage, the work allows for more individualistic tendencies: yet it does so via a diminishing 
of the experience of the work and the worlds of the creatures. The result is that there is not 
much affective 'moving together'. Here we can see that, even though Intimate Transactions 
in its final incarnation adopted features similar to many 'multiplayer game engine[s]' and 
competitive computer games, such as avatars and the principle of 'collecting. ..objects' 
from the creatures (Armstrong, 2005), the work is structured so as to draw the participants 
away from many of the regular outcomes and modes of engagement involved in much 
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gaming. For a start, Intimate Transactions is not competitive (Armstrong, 2005). Also, 
instead of a destructive individualism, as we have seen, in the more rewarding phases of the 
collaboration a kind of transversal group Eros is at work. A 'co-affective' drive vibrates on 
our bodies, destabilises established, often narcissistic or destructive habits and leads to a 
reparation of the virtual screen-worlds. 



This is very different to my own first experience of an interactive virtual game environment, 
back in the mid 90's. The game was Pterodactyls. We paid a fair amount of money for about 
three minutes spent in two different support systems, wearing large helmets and holding 
handheld controller-guns. Our handheld controllers could shoot things — Pterodactyls or 
any other moving creatures, including, as it turned out, each other's avatars. I couldn't see 
anything. As an unexperienced gamer, I didn't know what to do, feeling disoriented, I was 
just standing around. My co-player saw this. Thinking that I would know where I was if I 
turned around, he called out to me to do so. I did — physically and in the game. However, 
the context of years of conditioning took over and my avatar was shot. My co-player (a 
more conditioned game player) thought that I would get a few more lives but my avatar 
fragmented and failed to reappear (see also Murphie, 1997: 738). For the last few minutes 
of the game the other player was a lonely avatar looking to uncaring pixelated Pterodactyls 
for company. In Pterodactyls there is little 'ethical potentiality' (little 'relational difference in 
co-emergence'), little collaborative group Eros or 'belonging-together' — the power of 'co- 
affective' metramorphosis is severely diminished. 



The comparison between Intimate Transactions and Pterodactyls is stark. Intimate 
Transactions encourages collaboration in order for us to enrich the virtual screen 
environments and the gallery spaces (Hamilton, 2006: 128). One is not alone — the world 
vibrates 'with the power of difference' on the core of the body. Relation — 'difference between 
us' — is felt within and beyond the visual world (Ednie-Brown and Mewburn, 2006: 87). 



The Virtual/ "Interaction'VRelation 



A more philosophical conceptualisation of the virtual (one in which the virtual is not reducible 
to "VR" or the 'visible world') becomes important here (Murphie, 1997: 715; Guattari, 1995: 
91). For Massumi, the virtual is that which holds 'relational potential'. Moreover, the virtual 
holds this 'relational potential' in excess of what 'actually occurs' (Massumi, 2000a: 202; 
2002: 110). As we have seen, Ettinger's concept of the matrixial is a way of understanding 
how this 'relational potential' can take on an ethical quality. The matrixial expresses 
'relational difference in co-emergence'. This 'relational difference' is ongoing because the 
'relational potential' of the work is not exhausted in what 'actually occurs'. 
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As we have seen, the matrixial, and with it metramorphosis, express 'relational difference in 
co-emergence'. If Intimate Transactions could be seen as a metramorphic work, then it is a 
work that enables 'relational difference' not only to emerge but to replenish itself by being 
open to that which exceeds what 'actually occurs'. In short, the metramorphic processes in 
Intimate Transactions suggest a 'relational potential' that is never exhausted, even by the 
'transactivity' that takes place within the actual work. 



At the same time, as Massumi points out, '[any particular] potential does not pre-exist its 
emergence' rather, it 'comes into being, as becoming' (Massumi, 2000a: 192 and 201). There 
is a specificity to particular events of encounter. Each encounter within Intimate Transactions 
is a singular and unique actualization of the 'relational potential' — virtuality — in which the 
work has been created to immerse itself. This is despite the fact that each encounter with 
the work takes place in the same (technical) setting. Guattari suggests that it is here — with 
regard to the 'virtual' and not just the actual and 'visible world' — that various art forms, 
including "interactive" work like Intimate Transactions, 'have an important role to play'. Such 
work can participate in a recasting of the 'the axis of values' (Guattari, 1995: 91), precisely 
in terms of a concern with the 'relational potential' involved. For Guattari 'an ecology of 
the virtual is' thus 'just as pressing as ecologies of the visible world' (Guattari, 1995: 91). 
However, if we are concerned with relation and the virtual, as thought in the terms above, 
then the notion of "interaction" also needs to be reconfigured (Massumi, 1997a; 2000a and 
2011). 



Andrew Murphie has written about interactivity in terms of 'interactive becoming', which 
involves a taking account of the virtual (Murphie, 2005: 32; 1996), and Massumi suggests 
that '[w]e... translate the concept into relational terms' (Massumi, 2011: 67; see also Brunner 
and Fritsch, 2011 and Fritsch, 2011). [15] 



Massumi notes that 'interaction' has at times been conceived as 'a going back and forth 
between actions, largely reduced to instrumental function'. Yet this kind of instrumentality — 
present in the game Pterodactyls, for example — does not fully allow for the potentiality 
of relation and, as with my experience of the game, the work too easily loses its intensity 
(Massumi, 2011: 46-52). 



In short, if one is concerned, as in Intimate Transactions, with artworks that are "eco-ethical" 
(see Armstrong, 2006), 'it is... not enough' to 'simply' celebrate "interactive" work. Rather, 
attention must be paid to 'what modes of experience' the work creates (Massumi, 2011: 48). 
Massumi argues that with a shift from thinking in terms of interaction to 'thinking/feeling' in 
terms of relation there is also a shift in focus onto the 'relational potential' of the situation 
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the work both creates and immerses itself within [16]. Massumi suggests that it is when 
'interactive art' takes 'a situation as its "object"... not a use. ..not a behaviour.. .not a action- 
reaction' that it may become more than a game — possibly art (Massumi, 2011, 52-53, 47 and 
78). The work 'can take a situation and potentially "open" the interactions it affords'. This 
implies 'that the relational potential it tends-toward appears' (Massumi, 2011: 52). 



As we have seen, Intimate Transactions takes 'a situation as its "object"', not just the 
"interactions" between 'discrete elements' or bodies (Massumi, 2000a: 191). [17] In fact, 
Birringer considers Intimate Transactions in terms of the 'situation' of an entire 'transactive 
environment' (Birringer, 2006: 109). This is an environment that aims to keep the 'relational 
potential' of the work alive. As art (again, this is particularly obvious in the vibrations and 
the sound), Intimate Transactions 'tends-toward' a 'care for belonging' that concerns but 
exceeds the human participants. This care emerges most obviously at the stage of the work 
when there is a restorative and collaborative metramorphosis of bodies and creatures. 



In fact, in Intimate Transactions, there is usually more than one situation involved. A number 
of ongoing individuations are 'interlaced' (Ettinger, 1999: 18). It is in the work's commitment 
to a fostering of engagements- between these multiple situations that we can understand 
the relation between affect and 'transactivity' (see also Hamilton, 2006). For Massumi, affect 
moves between and connects situations (Massumi, 2000a; 184-185) — the two gallery spaces 
in Intimate Transactions provide a very clear example. As mentioned previously, affect is 
'multi-polar', 'transitivism and 'co-creative'. It concerns sociality across situations rather 
than the personal (Massumi, 2000a: 178-182). In Massumi's terms 'affect is transsituational' 
(2000a: 185). It can deterritorialise the local screen environments and move toward a 
transsituational 'shared world' (Birringer, 2006: 107). In Intimate Transactions, as we have 
seen, this is a world based on 'co-affective' collaboration and care for difference. 



Ednie-Brown has drawn on Massumi's notion of affect as trans-situational in her 
conceptualisation of the importance a 'trans-situational sensitivity' to the kind of process 
and collaboration involved in Intimate Transactions. She writes — 



Trans-situational sensitivity dislodges the 'contextual' from the assumption that one 
stands back and observes things 'as-they-are', as if there is a fixed worldly essence to 
be found. Similarly, it erodes navel-gazing or self-absorption in which the kingdom of 
the self and its expression reign insensitively supreme. (Ednie-Brown, 2007: 328) 



52 FCJ-149 fibreculturejournal.org 



Lone Bertelsen 



It is this kind of sensitivity that emerges in/through Intimate Transactions (see Ednie-Brown, 
2007: 327-328). 



In order to fully understand the importance of a 'trans-situational sensitivity' to 'co-affective' 
collaboration and process (rather than individual action) I will now give a brief account of 
Massumi's conceptualisation of affect as trans-situational. 



Transsituational Affect 



Massumi's develops the notion of affect as transsituational in relation to the 'experience of 
colour'. His starting point is an experiment from 1911 where the subject of the experiment 
is asked, based on recollection, to 'match' the colour of a 'friend's eyes'. In the experiment 
the recollected colour is nearly always more 'saturated' than the actual eye colour — it is 
'too-blue' (Massumi, 2000a: 178). This 'excessive' experience of colour is excessive in that it 
exceeds 'personal' memory or emotion. Massumi speculates that the experiment 'staged. ..a 
co-functioning of language, affect and memory' that is 'situational' rather than 'personal' 
(2000a: 178-182). In short, for Massumi the exaggerated recollection of the colour is in 
part due to the 'situation' of the experiment (2000a: 179 and 189). In this situation, the 
actual 'remembering of the colour is not effectively a reproduction of a perception, but a 
transformation or becoming of it' (Massumi, 2000a: 180). As it is a situational becoming — 
related in part to the experiment itself — toward a 'too-blue', the affect involved cannot be 
reduced to personal emotion (Massumi, 2000a: 184). 



Massumi goes on to explain 'emotion' in terms of 'personalized content' while he uses 
'affect' to conceptualise 'the continuation' beyond the realm of the personal (Massumi, 
2000a: 185). Again, the recollection of a friend's eyes as 'too-blue' involves a situational 
'excess' of colour that cannot be reduced to 'personal feeling'. Thus, for Massumi affect 
('the logic of affects') is both 'pre-personal' and also 'continues' after the individuation of 
feeling. This is to say it is also 'post-personal'. Affect involves a kind of 'presence of process' 
(Massumi, 2000a: 185). As we have seen it is this 'presence of process' that is emphasised 
within the design and experience of Intimate Transactions. Both the different screen 
environments, and the complex cross-sensory sound/vibration design, register the constant 
process of 'transactivity' across the two gallery spaces (Birringer, 2006) 
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This can be taken a little further. Massumi explains that the 'continuation' of the 
'personalized content' beyond the personal involves the context of emotion entering the 
'relational situation' of affect (Massumi, 2000a: 185 and 199). 'Affect is vivacity of context: 
situ-ation. Affect enlivens'. Context on the other hand processes a 'relative stability' in that it 
'pre-exists' (Massumi, 2000a: 187 and 181). 



In Intimate Transaction affect, even in the terms of excess described here, is expressed in 
the situation of the environments on the screens. This situation is the enlivening of the more 
or less stable contexts (technical and otherwise) in which the experience of the work takes 
place. However, Intimate Transactions is also 'transsituational' because there is a deliberate 
bringing together of multiple situations, (expressed, for example, in the sound, the shared 
screen-world and the vibrations). This only begins with the two different situations of the 
galleries. This deliberately designed possibility for (networked) transsituational collaboration 
can deterritorialise the more destructive habits of the individual (or, one could say, of the 
'context') (Massumi, 2000a: 185). Although, in one sense, the technical design of Intimate 
Transactions is clearly quite stable, designed as a repeatable 'context', in another way 
the design of the work quite deliberately 'tends toward' an opening to the 'transsituation' 
(Massumi, 2000a: 185), to relational potentiality, affect (and the trans-subjective). Massumi's 
suggestion that 'affect is transsituational' becomes very useful here (Massumi, 2000a: 185). 



In Intimate Transactions the 'co-creative' meeting in the worlds on the screen, the gallery 
spaces, the sound, the movement and the vibrations on the body could all be understood in 
terms of such an affective transsituationality. Thus Ednie-Brown's focus on 'trans-situational 
sensitivity', rather than 'context' (Ednie-Brown, 2007: 327-328). 



It is in particular in the excessive and unexpectedly intimate felt vibrations between 
situations that affect 'continues' and moves toward the 'transsituational' (Massumi, 2000a: 
185). One could say that in their 'excess' these vibrations 'will overspill' or 'escape' and 'enter 
other situations' (Massumi, 2000a: 184-185). In fact, Massumi suggests that the 'context- 
rocking transsituational drift' of affect 'holds the world together'. It 'is the life-glue of the 
world — a world capable of surprise' (Massumi, 2000a: 185-187). 



So to sum up, the joint 'co-creative' world in Intimate Transactions 'continues' and becomes 
'transsituational'. As discursive content, it comes to be. As excess, it continues' (Massumi, 
2000a: 185). The 'co-creative' world becomes transsituational within the work and, just as 
importantly, beyond it. The latter is another aspect of the ethical potentiality of Intimate 
Transactions. The work is capable of creating a care that lingers beyond the actual 
experience itself, as the ongoing possibility of changing modes of living and relating (see 
Armstrong, 2006: 16). 
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It is again important to emphasise that it is the excessive aspects (the vibrations, to take 
only one example) that participate in, to some extent enable, the individuation of new 
modes of relation. For Massumi such 'excess' allows for a relationality that is 'not reducible' 
to 'personalized' emotion. Rather, as affect, this excess involves 'movement', 'inhabits... 
passage' and concerns a 'joint situation' (Massumi, 2000a: 219, 185 and 181). 



For me, another good illustration of such an excessive affect — a 'continuation', which moves 
'personalized content' toward a 'joint situation' (Massumi, 2000a: 181) - is Alice's encounter 
with the Cheshire Cat in her adventures in Wonderland. In Wonderland, Alice comes across 
the cat a number of times. The first encounter is described as ending like this: 



"Did you say 'pig', or 'fig'?" said the Cat. "I said 'pig,'", replied Alice; "and I wish you 
wouldn't keep appearing and vanishing so suddenly: you make one quite giddy!" 

"All right," said the Cat; and this time it vanished quite slowly, beginning with the end 
of the tail, and ending with the grin, which remained some time after the rest of it 
had gone. 

"Well! I've often seen a cat without a grin," thought Alice; "but a grin without a cat! 
It's the most curious thing I ever saw in all my life!". (Carroll, 1960: 90-91) 

Engaging with Lewis Carroll, Deleuze writes that his books 'let an incorporeal rise to the 
surface like a mist over the earth, a pure "expressed" from the depths: not the sword, but the 
flash of the sword, a flash without a sword like the smile without the cat' (Deleuze, 1997: 22). 



In a sense this affective 'smile' is 'pre-personal' — a kind of 'pre-personal' "animality". Yet it 
is also 'post-personal', the 'continuation' of the 'personalized content' beyond the personal, 
the context of emotion entering the 'relational situation' of affect (Massumi, 2000a: 185 and 
199). If we follow Deleuze and Guattari, it could be said that Intimate Transactions, as in 
the meeting of Alice and the cat, extracts 'a consistent event from the [context and] state 
of affairs - a smile without a cat, as it were...' (Deleuze and Guattari, 1994: 126). Again, any 
situation, such as that of Alice and the cat, is always a meeting of situations. 



I would suggest that it is through a becoming 'transsituational' — through a 'moving together' 
as in Intimate Transactions — that the vibrations surprise and linger. Like the smile of the cat 
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they can take us to Wonderland, or at least toward wonder. This wonder provides a further 
ethical potential for 'transactivity' in art. [18] 



Massumi argues that "affect" and the kind of 'moving together' discussed so far — a 'joint 
situation' — involves a 'relational' or 'quasicausal openness'. It is concerned with 'sensing 
something new' (Massumi, 2000a: 193). He explains: 



Relationality cannot be accounted for by the objective properties of the actual 
ingredients in play considered as discrete elements. It cannot even be reduced 
to the interactions that may logically be predicted according to those properties 
... Relationality pertains to the openness of the interaction. Rather than to the 
interaction per se or to its discrete ingredients. (Massumi, 2000a: 191) 
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Figure 7: 'Inside Instability'. Image by Keith Armstrong. 
Source: http://www.embodiedmedia.com/, with permission of 

Keith Armstrong 



Relational Causality and Wonder 



Lone Bertelsen 




Figure 8: 'Shared body group'. Image by Stuart Lawson. 
Source: http://www.embodiedmedia.com/, with permission of 

Keith Armstrong 



This 'openness of the interaction' — its sociality, rather that the 'discrete ingredients' — is 
echoed in Ettinger's concept of metramorphosis. 



The experience of this 'openness' comes 'to the fore' in Intimate Transactions (Massumi, 
2011: 45). It is perhaps for this reason that, although one doesn't feel unsafe in Intimate 
Transactions, one feels a little uncertain and a little fragile. Surprised by the vibrations, 
for example, one wonders exactly what is going on and where one is (see Birringer, 2006: 
109). Yet Intimate Transactions encourages one, in such wondering, to be like the traveller 
discussed by Whitehead. Whitehead wrote: 'A traveller, who has lost his way, should not 
ask, Where am I? What he really wants to know is, where are the other places' (Whitehead, 
1985: 170). It is this kind of question, involving wonder and surprise, rather than certainty, 
that will keep the future alive (see Massumi, 2000a: 203-205). However, it is again important 
to remember that, as in Intimate Transactions, wonder does not primarily emerge from a 
'personalized' feeling or attitude. Rather, if we follow Irigaray wonder is A third dimension. 
An intermediary. Neither the one nor the other' (Irigaray, 1993: 82). [19] Drawing on Irigaray, 
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Ziarek explains that wonder 'operates as a transformative interval' and 'produces a change 
not simply in the manner of the subject's being but in the very mode of the relation itself 
(Ziarek, 1999: 6). Wonder is here conceived in terms of relation. It does not emerge from any 
individual "being". Rather, it is a constitutive force (Irigaray, 1993; Ziarek, 1999). 



In Intimate Transactions, if the pull of the work is followed, one does not get much of a 
feeling of being an isolated individual in control of the screen-world and its creatures. As 
mentioned, individual control is not encouraged by the very design of the work. There 
is, for example, no straightforward identification with an unchanging figure in an image 
world. Armstrong points out that throughout the engagement with the work 'a cascade of 
audio-visual and tactile feedback ripples back and forth through the server. This results 
in continual changes in the fluidity and movement qualities of the avatars' (Armstrong, 
2006: 33). Furthermore, as we have seen, the participants in Intimate Transactions are not 
encouraged to control the world through individual force. Rather what enlivens the work is 
an assemblage of forces - a networked, 'collective' or 'relational movement' (Birringer, 2006: 
112; Manning, 2009: 29). 



In games like Pterodactyls, however, one experiences a fairly clear relationship between 
"self" and avatar, and between subject and object. The aim is to be in control of anything 
that is "not-me" (the two players have guns for shooting anything "not-me", most obviously). 
In such "interactive" games there is not much space for a 'relational difference in co- 
emergence' that will 'restore' the 'virtual world' and keep it alive (O'Neill: 2006: 41). 



As opposed to many standard interactive games then, Intimate Transactions gains its 
intensity through a wonder that emerges from the complexity of the meeting of bodies, 



sound, creatures, screen worlds, vibrations and the two different gallery spaces. This wonder 
leads toward the 'new' (Irigaray, 1993: 75), not only within the experience of Intimate 
Transactions but, as suggested, beyond it. Here we can briefly turn again to the work of 
Massumi. 



Massumi's concern with change and the new is linked to both the possibility for surprise 
and to wonder (see also Irigaray, 1993). Like Whitehead (1968: 168) he discusses wonder 
in relation to philosophy and explains that philosophy 'prolongs wonder' because (as in 
Intimate Transactions) philosophy 'works... "against the stream of perception" as Bergson 
used to say, towards relationality "in itself"; towards the virtual' (Massumi, 2000a: 203). 
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Figure 9: 'Force of Change', Image by Stuart Lawson. 
Source: http://www.embodiedmedia.com/, with permission of Keith Armstrong 



For Massumi — 



Wonder is pre-philosophical, in the same way that habit is pre-scientific. Science 
formally prolongs habit (the reception of the new in an a priori mode of recognition). 
Philosophy speculatively prolongs wonder (the remainder of surprise persisting across 
its a priori capture by habit). (Massumi, 2000a: 205) 

Certain habits may be problematic. As mentioned these more problematic habits are given 
room in the phase of Intimate Transaction when objects can be removed from the creatures. 
They certainly dominate the interactive game Pterodactyls. However, as the 'activity 
dedicated to keeping wonder in the world' philosophy, (like the more collaborative phases 
in Intimate Transactions, when participants come together to restore the worlds of the 
creatures and the gallery spaces), can avoid an arrest by more troublesome habits and 'take 
a situation' (Massumi, 2011: 52) toward 'the [becoming] of a relation' (Massumi, 1997a: 203; 
see also 2000a: 203). 
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Wonder. This is where philosophy comes in. Philosophy is the activity dedicated to 
keeping wonder in the world... Philosophy then starts with the accompaniment: the 
perceived effects of relational quasicausality. It starts with the glow. Or the too — of 
the blue. (Massumi, 2000a: 203) 

Or perhaps it starts with a 'relational' and 'quasicausal' vibration on the stomach or lower 
back. For Massumi 'relational quasicausality' is the kind of causality that appears when 
the subject/object division is broken down: A quasicause has to be understood as referring 
'to effects that can only be explained relationally' (Massumi, 2000a: 202). In Intimate 
Transactions, for example, causality is no longer unidirectional (not that it ever really is). 
Rather, it is relational. Causality and with it responsibility become distributed (networked), 
emerging from an affective trans-subjective field. This is a causality that emerges from the 
relation itself ('transsituation' rather than 'context' [Massumi, 2000a: 185]). It thus cannot 
be discussed as a clear division between cause and effect associated with a subject and an 
object. Rather 'relational quasicausality' emerges from and keeps the 'relational potential' 
of the work alive. It enables something new to emerge across and between situations 
(Massumi, 2000a: 202). 



Massumi explains that '[classical cause concerns context' while 'quasicause concerns 
situation. Classical cause is reactive, in other words active-passive' while '[q]uasicause 
is sensitive-affective, or creative... .it expresses a real material reserve of unpredictable 
potential' (Massumi, 2000a: 192). 



'[Unpredictable potential' is expressed in the 'co-creative' and collaborative phases 
of Intimate transactions. With this affective 'co-creation' it becomes possible for the 
participants in Intimate Transactions to 'reopen their becoming-together to a relational 
quasicausality' (Massumi, 2000a: 206). In Intimate Transactions a more 'relational 
quasicausality' 'becomes expressive' (Deleuze and Guattari, 1987: 315) not only in the 
vibrations on the body or in the sound but within the complexity of the entire situation (see 
also Massumi, 2000a: 193). [20] The 'transsituation' itself 'becomes expressive' and a new 
'[trans]subjective music' is literally 'composed' (Guattari, 1996: 267, See also Webster, 2006: 
70). 

Diagrammatic Transactivity' 



How might this leave us with a more general approach to designing for relational 
'transactivity' in new media art and design? One way of answering this question may be to 
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ask whether transsituational affect can be diagrammed, and if so, how (see Ednie-Brown, 
2007: 327)? Ednie-Brown proposes the concept of the 'affective diagram'. For Ednie-Brown 
(who draws on Deleuze here) the '[affective] diagram is an assemblage of relations wherein 
the power to affect and be affected is distributed' (Ednie-Brown, 2007: 178). She further 
suggests that 'trans-situational sensitivity is also a sensitivity to the affective diagram' 
(Ednie-Brown, 2007: 327). However, '[t]he affective dimensions and sensual experience 
of relations — or that which constitutes affective diagrams — tend to be repressed under 
the mantle of representation' (Ednie-Brown, 2007: 198). The diagram must therefore be 
conceived of in more dynamic terms than as a static representation (see Watson, 2009: 11; 
Munster, forthcoming; Ednie-Brown, 2007: 198). [21] 



Here we can consider an approach to the diagram outlined in Guattari and Deleuze's work. 
[22] For Manning, who follows the thought of Guattari and Deleuze in a move beyond the 
representational, the 'diagram is not content driven' — that is, it does not work with context 
and signification. Rather, the diagram 'operates at the interstices of composition where the 
virtual is felt as a force of becoming' (Manning, 2009: 125). Similarly, Watson explains that 
for Guattari, the diagram is not a 'static image'. Rather, the diagram is a generative 'site 
of production' — a transformative and 'dynamic force' (Watson, 2009: 11-12). The diagram 
then does not only concern the human subject: it concerns the 'force of becoming'. Genosko 
writes that with the diagram Guattari seeks to escape the interpretative, 'meaning'-driven 
search of more representational frameworks and the 'human and individuated subject' 
(Genosko, 2009:103). With this move Guattari 'separates the image and the diagram: the 
former belongs to symbolic semiologies, and the latter to a-signifying semiotics' (Genosko, 
2009: 102). 



A-signfying semiotics are those: 



... that tune in directly to the body (to its affects, its desires, its emotions and 
perceptions) by means of signs. Instead of producing signification, these signs trigger 
an action, a reaction, a behaviour, an attitude, a posture. These semiotics have no 
meaning, but set things in motion, activate them. (Lazzarato, 2006) 

In sum, the a-signifying concerns affect and situation while signification concerns meaning, 
content and perhaps also context (see also Genesko, 2009: 99-105). As an 'a-signifying 
semiotic' then, the diagram concerns the kind of 'trans-situational' affect emerging in 
Intimate Transactions. It renders it perceivable and felt (see Ednie-Brown, 2007; Massumi, 
2011). However, as we have seen in Intimate Transactions the 'co-affective', constitutive 
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encounters are not only rendered perceivable and diagrammed in an image-world, they also 
traverse the different media forms. Engagement with the work is designed so that it involves 
a collaborative 'moving-together' of bodies and creatures that is diagrammed between 
screens, sound and felt vibrations. [23] 



Intimate Transactions then provides a very complex series of diagrams of the 'relational 
potential'/force of the entire 'transactivity' between bodies and 'across situations' 
(Ednie-Brown, 2007: 327) — the two gallery spaces for example or the different 'screen 
environments'. [24] 



We can clarify this further. The diagram has at least two sides. First, there is the more 
obvious 'diagramming' (Ednie-Brown, AG3, 2010) in the design of the work. This involves 
the diagrams that are created before the actual construction, along with the way that these 
morph into the energising structure of the work itself. Second, there is active 'diagramming' 
by participants within the work. The diagram in this sense is the affective coherence found in 
the ongoing transactions between Intimate Transactions as a work, and participants' (trans) 
actions, a literal 'diagramming' in situ. Of course, the lines between these numerous aspects 
of the diagram are sometimes blurred. Ednie-Brown suggests that 'the use of the word 
diagram can be confusing. ..but really the distinction is between diagram (charting/drawing) 
and "diagramming" (the act)' (Ednie-Brown in AG3, 2010). [25] Intimate Transactions is then 
a complex 'diagrammatic' work (see Deleuze and Guattari, 1987: 142; Ednie-Brown, 2007 
and Munster, forthcoming). It is diagrammatic not only in the quite obvious way, expressed 
in the functional structuring of transactions in designing the work — numerous diagrams were 
drawn in this process (O'Neill, 2006: 40). It is also diagrammatic in the sense of the diagram 
understood as that, which collects the changing relations — 'transactivity' — and keeps the 
potentiality of the work alive. 



Overall, the 'diagramming' in Intimate Transactions engages relational encounters and 
'transactivity' by operating within an intimate 'logic of affects' — thus Ednie-Brown's notion of 
the 'affective diagram'. A 'caring for belonging' emerges. This care — which is an ecological 
care tending to difference — is the very 'relational potential' Intimate Transactions 'tends- 
toward' (Massumi, 2011: 60). It is hoped that this care will follow us when we leave the 
gallery spaces. 
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Notes 



[1] The other members of the Transmute Collective are Lisa O'Neill ('performance director') 
and Guy Webster ('sound director') (see Armstrong, 2006: 16). Intimate Transactions was 
created collaboratively and involved the creative work of numerous other collaborators. For 
the names of all these collaborators see http://www.embodiedmedia.eom/#/page/inti mate- 
transactions . See also Armstrong's (2005) own article on Intimate Transactions published in 
the Fibreculture Journal. 



[2] The work is now a part of the permanent collection of ZKM media arts museum in 
Karlsruhe, Germany. 



[3] Jillian Hamilton also considers the ethical qualities of Intimate Transactions (see 
Hamilton, 2006: 129). 



[4] Barbara Bolt writes that for Birringer (as for others) 'the critical element in Intimate 
Transactions is its capacity to raise questions to do with the ways we think about and 
intervene in the world through our (design) practices' (Bolt, 2008: 28). 
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[5] Zeljko Markov who designed the "shelf" explains that the 'immediate challenge was to 
find a way of supporting the human body in a neutral position that' was 'not too familiar and 
yet not threatening' (Markov, 2006: 45). 



[6] Greg Hooper writes that 'the whole system forms an ecology and we have avatars within 
the system: jellybaby angels or glowing discs floating submerged in a dark ocean' (Hooper, 
2005: 26). 



[7] Italics my emphasis. 



[8] Guattari explains that the '"term collective" should be understood in the sense of a 
multiplicity that deploys itself as much beyond the individual, on the side of the socius, as 
before the person, on the side of pre-verbal intensities, indicating a logic of affects rather 
than a logic of delimited sets'. Guattari here also points to the 'incorporeal Universes of 
reference such as those relative to music and the plastic arts'. For Guattari ' [t] his non-human 
pre-personal part of subjectivity is crucial since it is from this that its heterogenesis can 
develop' (Guattari, 1995: 9). 



[9] Armstrong (2005 and 2006) gives a detailed outline of the ecological and 'ecosophical' 
commitments underpinning the design of Intimate Transactions. He draws extensively on the 
work of Guattari. 

See also Genosko (2009) for a discussion of ecosophy. 

Hooper writes that '[Intimate Transactions] continues Armstrong's development of 
ecosophical praxis, used here as a pragmatic philosophical take on new media production 
that chucks out the techno-fetish and puts in a fusion of ecological theory and ethics. New 
media as experience design rather than commodity production' (Hooper, 2005: 26). 



[10] In her engagement with Simondon, Manning explains 'that the force of affect resides' 'at 
the virtual-actual juncture' and that it is affect that 'returns, not the subject' (Manning, 2010: 
117-126). 



[11] See also Birringer (2006) and Manning (2009). 



64 FCJ-149 fibreculturejournal.org 



Lone Bertelsen 



[12] See Bertelsen (2004) for a discussion of this. See also Ettinger (2006). 



[13] Italics my emphasis. 



[14] It should be noted here that Genosko also writes that Guattari's 'choice of Eros entails the 
group subject, that is, a definition beyond the traditional dualities of society and individual, 
Eros and Thanatos' (Genosko, 2000: 156). Arguably Intimate Transactions also moves in this 
direction. 



[15] Christoph Brunner and Jonas Fritsch (2011) have introduced the Simondonean concept 
of transduction into the discipline of interaction design and Fritsch (2011) has activated 
Massumi's concepts in a detailed (re)thinking of the discipline and affective 'interactive 
environments'. This work is very much in tune with the concerns of this article. 



[16] Massumi could be thinking of Intimate Transactions when he writes: 



'you've built into the operation shifts in emphasis from interaction to lived relation. 
You're creating ways of making lived relation really appear. You're operating on 
the qualitative level of thinking/feeling, where you are pooling styles of being and 
becoming, not just eliciting behaviours' (Massumi, 2011:52). 

[17] Birringer points to how site-specific performance too was conceived as 'forming a 
situation' (Newling in Birringer, 2006: 107). He suggests that '"transactions" reverberate with 
much of what we remember in the history of live art in which bodies are placed in situations' 
(Birringer, 2006: 107). 



[18] In the world of thinking interaction design Ednie-Brown and Mewburn suggest that 
laughter, and I would suggest with it the like of the grin of the Cheshire cat and the vibrations, 
are quite significant here (see, Ednie-Brown and Mewburn, 2006: 85-86). They keep the 
intensity of the relation alive. 



[19] For Irigaray wonder is '[i]n us and among us' (1993:82). Wonder must always be '[f]aithful 
to becoming'. And it 'is an openness across, awaiting new insights' (Irigaray, 1993:75). 
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[20] In Ednie-Brown and Mewburn's (2006) terms Intimate Transactions vibrates with the 
'relational potential' between us. If we follow Massumi, in this vibrating trans-situationality, 
causality becomes "relational" (Massumi, 2000a). Causality cannot be prescribed to any 
individual body or element of the work alone but emerges from a dispersed 'co-affective' 
movement. 



[21] Anna Munster in her forthcoming book (An aesthesia of networks: conjunctions of 
experience, media and art) considers how networks experience. Simply put Munster argues 
(amongst other things) that such experiences and much networked art are diagrammatic, 
relational and affective. She draws on the work of Pierce, James and Deleuze and Guattari, 
amongst others. See in particular the chapter titled 'Networked Diagrammatism: from map 
and model to the internet as mechanogram'. 



[22] See in particular Deleuze and Guattari (1987: 111-148 ); Deleuze (2003) and Deleuze 
(1986). See also Massumi (2002: 177-207; 2011, 83-103). 



[23] Guy Webster explains that the Transmute Collective aimed to make the participants 
'feel a part of the work. He explains that this is something that is 'very difficult to do with a 
screen. But you can do it with sound... [t]he sound and the vibration, rather than the visuals 
alone perform that function' (Webster, 2006: 60). 



[24] See Lisa O'Neill (2006: 40). 



[25] Ednie-Brown (AG3, 2010) writes that 'Brian Massumi's "biogram" is usually what I mean 
when I talk of diagram (or "affective diagram")'. See Massimi (2002: 177-207). See also 
Ednie-Brown (2000) for a discussion of the diagram. 
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AffeXity 



AffeXity is an enquiry into affect in cities, and a-fixity as an urban condition. It is an artistic 
research project, but really it is a set of overlapping practices: artistic practices of dance 
improvisation, video shooting, digital image editing and sound composition, combined with 
the daily practices of moving through a city and using mobile devices. Add to this bundle the 
applied technical research of developing applications for mobile devices and the practices 
of writing and reflecting on all of the processes, and you have an unwieldy assemblage. The 
entire project is animated by explorations of affect. It is in constant motion, exceeding both the 
artistic direction or conceptual coherence that attempt to structure it. [1] 



This project opens implications for interaction design: designing affectively and designing for 
affect are two different things. It is possible to do the latter using processes and methods that 
are entirely un-affective, or in other words without affective sensibility. The opposite might also 
be true: it is possible to use an affective design process for objects or experiences that are not 
affectively noteworthy. With AffeXity we are composing affective processes for the production 
of an affective experience. We do not know yet whether they are effective, but it is very likely 
that 'effectiveness' is not an appropriate criterion for judgement. 



At the heart of AffeXity is the convergence between performance and mobile technologies. In 
particular, the project uses the augmented reality browser Argon. This runs on smart phones 
and permits images, audio and video to be anchored in real world locations using geospatial 
coordinates or QR codes. The media float in the device's display over the live feed from its 
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camera. Appropriately for an augmented reality project, AffeXity is designed to have several 
layers, or modes, of interaction. In its simplest interactive mode, it is a locative media project 
with short movement improvisations videoed and geospatially tagged at various sites in 
Malmo, Sweden. These locative choreographies are shot in the location where they will 
be tagged, with the dancer improvising in a mini site specific performance. Once they are 
registered using GPS or QR codes, these embedded choreographies can be seen by people 
with mobile devices as they travel through Malmo in the courses of their lives. Whether it is 
an habitual route to work or walking the dog in the middle of the night, the choreographies 
can be downloaded and experienced at any time of day or night throughout the year as 
long as mobile coverage permits. These narratives can be as integrated into urban lives as 
crowds, buildings and buses, but perhaps offer an idiosyncratic 'shimmer' to experience, a 
play across presence and motion (Barthes, 2005: 101). 



A more orchestrated option for interaction is for small groups to be led from site to site by 
a guide, in the style of a promenade performance. With multiple devices downloading and 
playing the visuals and sounds at slightly different speeds the group will provide their own 
re-mix of the media simply by being co-located while viewing. An expanded performance 
option is planned for occasions when a more complex mix of presences can be formally 
produced, as in the case of a festival: the same live performers from the media will be 
situated in the sites where the locative choreographies are embedded and projections on the 
urban structures will accompany the imagery and sound on the devices. 



An additional performative layer for AffeXity takes it even further into the domain of social 
choreographies. This will occur when the project is promoted through social networking 
and other initiatives for contribution and participation by whoever desires to do so. Existing 
choreographies can be downloaded, remixed and retagged; new choreographies can be 
posted to exist in relation with others; sound, animation or text might be added, or other 
forms of embedded media might be offered. The exchange and circulation of affect through 
mobile devices happens all the time as we SMS, tweet and post to Facebook, but this 
project intends to foster an increasingly performative approach to media and urban living, 
broadening the choreographic patterns of daily life which risk being ever more controlled 
and narrowed. [2] 



This paper is written at a fairly early stage in the research process, contrary to the 
conventional scholarly practice of reaching closure on a project prior to reporting on it 
from the clear position of hindsight. This is done for two reasons: the first is that our early 
phases of research already have revealed considerable tensions and insight into affect and 
performance both practically and conceptually; the second, more compelling, reason is that 
this project has an existence prior to its completion as an artwork. It may take years for 
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the piece to end (it will never be complete or closed) but from very early in the process it 
generated a fever of interest, with blogs and cultural organisations reporting it as existing, 
resulting in a flurry of demands on the part of festivals and producers to host it. Viewed from 
the processes of production and reception in the art world this caused uneasiness: almost 
provoking us to correct the misconception that it existed when in fact it did not yet, making 
us want to slow its public reception, to decelerate the project, until we could catch up with 
it. Viewed affectively there is no inconsistency. According to the Spinozan formulation of 
bodies that is increasingly cited in critical and philosophical writing, AffeXity exhibits the 
capacity to affect and be affected, and it has its own patterns of speeds and slowness. [3] 
As a construction, as a body, it exists apart from whether or not it is fully complete according 
to the art world, or indeed according to the artists. Closure is irrelevant. The project will 
always exist in a state of not quite, or not yet. From its inception, this project created a sort 
of affective cloud around it, almost a microclimate, that was palpable and circulating. And so 
I write and think about it, more to catch up with it than to explain it. 



The content of this paper seems to flow in several directions at once (philosophical, artistic, 
technological, methodological...) so I'll take a moment to be quite pragmatic and indicate the 
path to be followed below. The discussion of AffeXity will be opened by referring to some of 
the philosophical currents around affect such as intensity, autonomy and contingency; then 
the AR browser Argon and the impact the technical development and the artistic processes 
have on each other will be considered. The middle section of the article will paint a picture 
of affect as it emerges from the actual practices that constitute AffeXity's early phases of 
artistic research: dance improvisation, video shooting, image editing and choreographic 
direction. There is an affective turn in each of these practices that does not necessarily map 
onto the affective turn in theory; this will be explained by relying on descriptions from the 
moments of practice by Jeannette Ginslov and myself, the main artistic collaborators. The 
final section is devoted to outlining a methodological perspective emerging through this 
research. Calling it provisionally affective sensibility, it straddles practical and theoretical 
activities because it is used in the process of generating the movement and media at the 
same time as it is a way of engaging with theories of affect. It is a method for generating 
artistic and theoretical content. What will not be discussed in this paper is the actual 
implementation, reception or, to use a design term, the 'user experience' of AffeXity, for the 
simple reason that the project is not yet at that stage of development. 



The reflections on affect and the description of the artistic research process in this paper 
can stand alone, but the intent is for them to contribute to a particular understanding of 
performance, that most 'promiscuous' of interdisciplinary practices and scholarly memes 
(Thrift citing Dolan, 2008: 124). Performance, in practice and theory, has sparked the interest 
of interaction designers, urban geographers, media studies scholars and architects in 
recent years. There is a sense that performance and the performative can shake the dust 



74 FCJ-150 fibreculturejournal.org 



Susan Kozel 



off conceptual constructions that seem too antiquated to account for the convergence of 
disciplines and practices, or that they that can provide a level of dynamism, imagination 
or simply liveness to methods, workshops and practices. Speaking from the perspective 
of scholarly work in interaction design, performance is frequently posited as a challenge, 
solution or method, and then walked away from without much elaboration. On occasions 
when it is elaborated, the usual suspects of Richard Schechner (1988), Erving Goffman (1967) 
and Victor Turner (1987) are relied upon to emphasise aspects of anthropology, ritual or the 
performances of daily life. In this paper performance is triangulated across bodily movement, 
emergence and shimmering. Performance as emergence occurs out of the fissures in 
habit and codified behaviour; it is heavily influenced by improvisation. Shimmer is based 
on change, not just change in position but a ripple in affective content and a flicker in the 
force of habits: it is 'the shimmering field of the body insofar as it changes, goes through 
changes' (Barthes 2005: 73). Performance can be seen as a play between the escape and 
recontainment of movement, as 'a shimmer of forces' of bodies and of things (ibid, 54). 
Never an easy concept to pin down, this understanding of shimmer will be unfolded further 
below, in particular when movement and video editing practices attempt to avoid what 
is habitual and drop into the unexpected. Ginslov uses the metaphor of 'sniffing out' the 
movement she wants to capture on video, the scent of something that is out of the usual 
patterns. Performance as shimmering has relevance also to social choreographies, seen as 
the many minute actions, uncontrollable from a central source, of a multiplicity of bodies and 
objects. 



Extending Affect 



Affect is notoriously hard to define, and this can make writing about it seem precious or coy, 
particularly if clear explanations are avoided. Suffice to say that any definition is not going 
to fit, but take this one in good faith: affect is the passage of forces or intensities, between 
bodies that may be organic, inorganic, animal, digital or fictional. It is located in the domain 
beyond reason, logic or 'conscious knowing' (Gregg and Seigworth, 2010: 1) but is so much 
more than emotion or feeling. Emotions are a jumping off point for understanding affect, 
the common currency of affect (Thrift, 2010), but the swirl of corporeal, conscious and pre- 
reflective forces that make up affect exceed the narrower domain of emotions. Melissa 
Gregg and Gregory Seigworth offer an excellent overview of affect, they manage to provide 
just enough clarity but not too much, just enough poetry without becoming too ethereal; 
and crucially for me, they do not extract affect too far from bodily experience. 'Affect, at its 
most anthropomorphic, is the name we give to those forces that can serve to drive us toward 
movement, toward thought, toward extension' (Gregg and Seigworth, 2010: 1). Extension 
in space and time through mediation and mobility is a possible way of describing the basis 
of AffeXity, but this extension is more than the revelation over the past decades, beginning 
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with telematics, that we can be present in distant locations and be recorded and replayed in 
different temporalities. [4] 



The extension relevant to current practices and discourses around technologies of presence 
and communication is more of a repatterning of the forces that make up our bodies and our 
lives. This repatterning is no longer just construed as movement extending outwards from 
a contained subject. Whether through recording and layering, or by means of the sharing, 
annotating and reposting of social media, this repatterning is not a vector: it is a complex 
choreography of extension and intensities, of externality and immanence. In designing and 
devising the performances that make up AffeXity, we are less concerned about the physical 
forms of bodies in urban spaces than the play of intensities radiating from and through 
people with their devices. 



Extension and retraction are familiar biomechanical motions in most dance forms (plie and 
tendue, bend and stretch). In taking a choreographic approach to affect, AffeXity locates 
these actions in the wider flows of the extension, compression, radiation, dilution and 
multiplication of affect. These are not simply qualities beginning or ending with a solitary 
performer or recipient of media: extensive relations and intensive capacities are located in 
a shared domain. When Deleuze wrote that extensive and intensive relations are not just 'a 
matter of utilizations or captures, but of sociabilities and communities' he, of course, was 
not writing about Augmented Reality (Deleuze, 1988: 126). He refers to ethology, the study 
of relations between things. However, his ideas are useful in that mobile media is not merely 
about what we can capture or whether we utilize our mobile phones 24/7. Deleuze helps 
redirect our focus to the creation, destruction and recreation of different social groupings or 
extended bodies. [5] 



Affective forces need not be forceful. They can be barely detectable shifts in relationality 
between ourselves and our built environments, or between bodies in urban spaces 
moderated and mediated by technologies. AffeXity, as a locative media choreographic 
project, explores a body's ongoing 'immersion in the world's obstinacies and rhythms, its 
refusals as much as its invitations' (Gregg and Seigworth, 2010: 1). In cities we encounter 
brick walls that refuse, and glass windows that invite; but equally the glass windows 
obstruct and the brick walls offer crevices for opening. The obstruction or invitation is not 
sustained, but the state of relations is. Affect is as much outside a body as within it, and 
ultimately clear distinctions between inside and outside no longer make sense. 
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Muddy autonomy 



As important as it was in the 1990s to suggest that affect was autonomous it is now equally 
important to emphasise that it is not separate from situated, messy exchanges between 
dynamic bodies. 



The autonomy of affect is its participation in the virtual... Affect is autonomous to 
the degree to which it escapes confinement in the particular body whose vitality, or 
potential for interaction, it is. Formed, qualified, situated perceptions and cognitions 
fulfilling functions of actual connection or blockage are the capture and closure of 
affect (Massumi, 2002: 35). 



Brian Massumi's formulation has generated many reactions but they tend to fit into two 
currents. Some readers are prompted to let out a sigh of relief that they can escape the 
discourse of bodies, while others dig their heels in and emphasise that affect is still corporeal 
and situated. Instead of being forced to situate myself on one or the other side of yet another 
pernicious duality (that is, is affect free from or situated in bodies?) I prefer to ask what is 
meant by bodies. How are they constructed, composed and lived? I find it surprising, and quite 
revealing of a particular intellectual climate, how many people — media students, musicians, 
dancers , literary theorists, not to mention designers of games, software and services — are 
relieved and even captivated when they hear of a Spinozan approach to bodies. The relief 
comes from no longer being cornered into choosing between the meat or the abstraction, 
but being able to live with the combination as both necessary and contradictory. [6] I choose 
to interpret Massumi as creating a sort of undulating current between affect within and 
outside my body, possibly because I have a strong corporeal resonance with his assertion that 
Actually existing, structured things live in and through that which escapes them' (Massumi, 
2002: 35). At the same time, I am wary that too much emphasis on autonomy falsely lulls us 
away from the awareness that we are corporeally impacted by affective practices. I breathe 
a little easier when I read Sara Ahmed's assertion, 'I do not assume there is something called 
affect that stands apart or has autonomy ...Instead I would begin with the messiness of the 
experiential, the unfolding of bodies into worlds, and the drama of contingency, how we are 
touched by what we are near' (Ahmed, 2010: 30). I appreciate the nuance of Eve Kosofsky 
Sedgwick and Adam Frank's writing on Silvan Tompkins, but as an artist attempting to work 
affectively I embrace even more their practical insights that, like it or not, affect is felt and 
that affective events afford the possibility not to be affected. They describe the act of reading 
Tomkins: At least as often as paragraphs permit reader and writer to do — here to enjoy but 
in other places to anger... — they permit one to not do' (Sedgwick, 2003: 96). This points to 
a latent political motivation for AffeXity. Urban dwellers are ever increasingly affectively 
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manipulated by political and economic forces without the scope to not be affected. This is 
echoed in another way by Thrift and Amin when they imply that we ignore affect at our peril 
(Amin and Thrift 2002). 



Affect exists. It is a part of me and beyond me. Affect is really affects: they are shared, 
and shaped by me and by others. They seem to hang in the air, they live in the pit of my 
stomach. They do both at the same time. There are strings attached. Sometimes they come 
from the marginal movements of banal bodies. I absolutely cannot access affect without 
my body but it does not reside or originate or remain in me. Affect does not discriminate 
between age, gender, materiality or bit rate. It is perceptually polymorphous, and socially 
ambiguous. Beyond taste and judgements. Affect is and is not at the same time. It is 
excessive and beneath notice. It is performative, shimmering. The moment of affective 
emergence exists in between minute movements and decisions, in the tension and the flicker 
of motion. 



Augmented Reality as Augmented Materiality 



'Considerations of affect are impossible to decouple from those of materiality,' writes Susanne 
Paasonen. Materiality in her research refers to the bodies performing in and viewing 
pornography, but it also refers to 'the technological objects, protocols, networks, and 
platforms' involved (Paasonon, 2011, 8). Her argument can be mapped with ease onto the 
experimentation with augmented reality as a platform for the transmission of affect across 
bodies that themselves exist across layers of mediatization. 



Argon, the augmented reality platform we use, is currently the first and only application to 
offer video as a layer, alongside text, audio, still images and the possibility for animation. 
[7] The development of AffeXity occurs synchronously with the development of Argon, 
prompting the question: Why work with a developing piece of software? 



As hackers and improvisers have long known (since the days of experimentation between 
artists and scientists such as E.A.T. in the 1960s and the wave of technology and dance 
experimentation in London in the 1990s called Digital Dancing) when something is not quite 
finished it has an immanent quality of unfolding: we don't quite know what functionalities 
it will have or how these might be used. This echoes Spinoza's oft-cited words on bodies 
'No one has yet determined what the body can do' (Gregg and Seigworth, 2010: 3). We 
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work with Argon because no one has yet determined what it can do. Like the performing 
bodies in AffeXity Argon is itself a body which exists in a state of potential. It is still being 
sketched, or to use Massumi's phrase it participates in the virtual (Massumi, 2002: 35). 
Two caveats need to be introduced to this somewhat Utopian tone of immanence and 
potential, one pragmatic and one political. The pragmatic one is simple: when a piece of 
software is not yet completed it is buggy, limited and frustrating to use. It crashes. The 
interval between what you imagine it might do and what it currently does is what provides 
momentum to develop further, but that interval also taunts you. 



The political caveat is that in our current social and economic climate very little software is 
created without already having a place in the consumer market. In their pamphlet on Urban 
Computing, Adam Greenfield and Mark Shepard accurately acknowledge that advertising is 
the biggest problem lurking behind urban computing (Greenfield and Shepard, 2007: 15-16). 
The projects they discuss enter into critical relationships with advertising and surveillance. 
Like most AR software, Argon is well positioned to serve the advertising industry, but it 
has some important distinctions. It is free, open source and operates according to open 
standards for web content. Further, the development team opts for breadth by emphasizing 
the potential for cultural heritage applications alongside more commercial ones and by 
working with artists to develop its potential for artistic use. No one has yet determined 
what this body, the non-human body of Argon, might do. Or what the bodies that are 
combinations of people, devices and software might do with Argon. 



More specifically, it is worth asking what the AffeXity artists seek from Argon, if we could 
have any functionality at all? In terms of affordances, our desires are not surprising. We 
want from our mobile media devices what most people do: rapid downloading; a fairly 
intuitive interface; the ability to manipulate our video images; a capacity for annotation 
so that one choreography can be linked in AR space with others, thus creating a network 
of images commenting on each other. This wish list of functionality can also be expressed 
in the terms of affect: we desire the ability to slide across speeds and slowness in order 
to modulate how we affect and are affected. In other words we want to have vectors 
of gradience built into the augmented reality application, or the ability to shift visual or 
temporal intensities by subtle degrees. Sliders work far better than buttons (as any DJ 
knows) and we would like several: one permitting us to adjust the transparency of the 
images, one to dissolve the edges of an image by degrees, and another to shift the size 
and rotation of the images. All of these can be considered technological affordances for 
the transmission of affect, or 'durational indices of shapes, timings, rhythms, folds, and 
contours' (Gregg and Seigworth 2010: 13) but they do not end with image manipulation. 
These also return to Paasonen's multiple designation of materiality by introducing the 
materiality of perception and the texture of the images (Passonen, 2011: 8). These are our 
desires, but they are a long way from being implemented. We are in the midst of that most 
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familiar but not-so-often-written-about phase of the technological 'work around', finding 
ways to work around limitations. 



The actual locative component of locative media needs to be reflected upon further. Geo- 
spatial tagging is not a new technology, but what happens when the tags are heavily 
populated — when multiple images need to inhabit a GPS location? Like densely populated 
cities, the layer of augmented reality itself contains layers. It should be called augmented 
realities. The plural form invites the choreography of parts, or 'relation as rhythm to account 
for the passage of intensities between bodies, or bodies and world (Gregg & Seigworth, 
2010: 13). Or the term Augmented Materiality would do: AM instead of AR. Deeper 
discussion of locative aspects will wait for a later paper once we deal with this phenomena 
more tangibly. Right now it is still speculative for us but we can see where current discourse 
stops short. For example, does AR have to be about the clear framing of media in our 
devices and legible insertion into cities? Or can it be used as a mode of peripheral vision, as 
considered by Juhani Pallasmaa in his reflections on architecture when he writes that what is 
missing from our dwellings today are the potential transactions between body, imagination, 
senses and the environment (Pallasmaa 2004)? Can we go beyond the 'choreography of 
sensations,' a formulation by another architect critically questioning his field, toward a 
performance of affect (Hague, 2003)? 



An Inventory of Shimmers - exploring the process 



Here the focus shifts to descriptions of the actual processes of performing affect. As such, 
this section is 'an inventory of shimmers, of nuances, of states, of changes' (Barthes, 2005: 
,77), revealing degrees of gradience between speaking from within movement and reflecting 
on movement. This 'inventory' can be seen as an accumulation of raw affective data from 
experience obtained by applying the method of affective sensibility. It can also be seen as 
an "affectual composition" (Gregg and Seigworth, 2010:11). The usual structure of scholarly 
writing is inverted (or contradicted) because the data in this section is presented just prior to 
the articulation of the method in the final section. Echoing the discussion above, this can be 
seen as a 'necessary contradiction', respecting the fact that the methodological perspective 
emerges at the same time as the artistic material. In the paragraphs below JG and SK 
are used to indicate the words of Jeannette Ginslov or Susan Kozel. We write and move in 
such a way as to explore the small shifts of affect as they make themselves felt in motion, 
perception and aesthetic habits. 
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Movement improvisation: 'Where to start?' 



SK: How do I even begin to think about working with affect? For so many years the 
improvisatory focus was on kinaesthesia, on corporeality and embodiment with 
all its multiple voices in motion. Here I stand in a city about to begin a movement 
improvisation. Where to start? I attempt to start from somewhere different from my 
usual starting points: I try to inhibit my habits. I have an inkling that I usually begin 
a kinaesthetic trajectory by releasing my arms and letting my hands travel, leading 
my body, at the same time as softening my knees letting my spine create a different 
torque. Now I decide to reframe the starting point: can I let affect lead me rather 
than motion? 

I want to move less and sense more. Sensing is not just an internal focus, it radiates 
outwards. When an affective state arises and is not fought or tampered with it acts 
as a window: as the mover, I can choose to notice this window and pass through it. 

Of course the entire notion of a 'starting point' is awkward and forced because dance is 
a pattern that occurs all the time. Improvisers of movement and music have a practical 
understanding of Deleuze's assertion that 'it is by speed and slowness that one slips in 
among things, that one connects with something. One never commences; one never has a 
tabula rasa; one slips in, enters in the middle; one takes up or lays down rhythms' (Deleuze, 
1988: 123). 



Yet, when working with media that begins with video capture there is a starting point, or at 
least a threshold: it is when Jeannette tells me the camera is rolling. 



JG: I am very aware of the task at hand: the desire to capture the affective 
gestures that the dancer is performing far outweighs the capture of movements or 
choreographies. I relax, breathe, but I am alert to all the subtle nuances. I try not to 
think too much or direct too much. This direction is a gentle persuasion. The dancer 
needs to sniff and tease these out by exploring her somatic connection to the space 
she is performing in. We do not think of dance, steps, counts, but enchai'nements of 
affect. There may be a score that is decided upon, something to work on. The dancer 
and I enter that resonant space. When I feel it is there, I hit the record button. 
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This is a subtle variation on improvisation because improvisation is about suspending 
conscious judgement and letting the flow of inner or outer stimulation direct the flow of 
movement, talking, singing, reacting, making and creating. It has some relation to contact 
improvisation. Contact improvisation has been characterized as cultivating alertness 
'in order to work in an energetic state of physical disorientation.' [8] In other words, 
improvisation is not just about fluidity and synchronicity. Consideration is given to inertia 
and disorientation, gravity and temporality, how contact with the ground, others or oneself 
may be varied, sporadic, and inconsistent but is all the stronger because of these qualities 
(Kozel, 2012). 



SK: In this particular improvisation I am standing in front of a lighthouse in Malmo. 
It is sunny and cold. People are passing close by, for this lighthouse is near a 
drawbridge leading to the ship building yards close to the university and many 
other buildings. What is the affective window? It is a combination of impulses from 
inside and outside: I bend my knees and fall over the railing wanting to dissolve 
into particles at the same time as turn to water on the stones. There are emotional 
overtones, but the affective state is more than feeling tired or a little anxious or 
happy to be outside in the air as the seasons change. I can't quite capture it, or seem 
to slide in and out of different affective currents. There are traces of the social urban 
environment for the presence of the two observing tourists and the cars, bicycles and 
trucks passing. Suddenly I am more aware of what is going on outside of me than 
inside: the tourists begin to stage their own performative shoot, letting themselves 
become more adventurous in how they use their bodies as they are co-located with 
us using our bodies in a way that is clearly for a film or art project. Two workers 
see me slide down the concrete slope to the rocks and water below and ask, partly 
humourously, partly in earnest, if I need help. The affective state is made up of the 
emotional, physical, social, environmental and meteorological. 




Video still from shoot at the lighthouse in Malmo. Photo Jeannette Ginslov. 
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http://www.youtu be.com/watch ?v=2Gf7ohn5SY8&feature=youtu. be 
AffeXity Phase 02 Lighthouse 01 

http://www.youtube.com/watch?v=ywfSTKEC3KI&feature=youtu.be 
AffeXity Phase 02 Lighthouse 04 

(The online version of this paper includes embedded video as cited above and is published at 
http://twentyone.fibreculturejournal.org/fcj-150-affexity-performing-affect-with-augmented- 
reality/ -ed) 



A way of understanding the different movement patterns in AffeXity is to say there is a shift 
from patterns to 'feeling tones,' to movement qualities or 'atmospheres' (Seigworth & Gregg 
2011: 2). This is why the cloud is an apt metaphor, not for the usual connotations of invisible 
data clustered around our heads, like a swarm of bees, but because clouds can be damp, 
dense, light, ridged or bubbly. Passing through a cloud or carrying one around with us cannot 
help but be affecting on the levels of sensibility, thought and motion. 



Watching the images from the lighthouse shoot, I see how 'in my head' I was struggling not 
to direct, but instead to release the tendency to control and direct movement. Jeannette 
too, when presenting this section before a small audience paused and said 'it's not quite 
right, it is not quite ... there.' [9] Of course not. Affect is never there, it implies an "extreme 
changeability" (Barthes, 2005: 101). Artistically there is the possibility that the images 
and sequences will never feel complete or quite right. At the time of writing this article 
we are still in the early stages of developing an improvisational approach but there is a 
possibility that when the motivating forces are affective there will never be a coherent 
aesthetic quality to the movement. It will always be about disorientations and attunements, 
in perpetuity, in and out of one's head. This is what makes this project especially suited to 
social choreographies, rather than the vision of an artist or collaborative team, where social 
choreographies are the multiple minute actions of a myriad of bodies, undirected by a central 
source (Kozel, 2010a). This project needs to live in the wider domain of social networked 
media, constantly being raveled and unraveled, ignored or obsessed over, by many people 
according to the different rhythms of their lives and bodies. 



SK: And next thing I know I am draping over the barrier and floating upward again, 
my body is moving of its own accord. I feel like a strange bird at the same time as 
embed the cold metal into my lower abdomen. The improvisation takes me into a 
standing arc with arms slicing the air, and reaching back with bent elbows. My back 
is important again. I realize, perhaps for the first time that when improvising vision 
shifts: no longer central and frontal, all visual processes seem to be peripheral, 
sensing in the round. 
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Video Capture : Intensities and Resonances 



With AffeXity the capture of movement on video is done with care, always with an 
awareness of what Pallasmaa has called the danger of reduction to a 'retinal journey.' He 
writes of architectural design and how 'computer imaging tends to flatten our magnificent, 
multi-sensory, simultaneous and synchronic capacities of imagination by turning the design 
process into a passive visual manipulation' (Pallasmaa, 2005: 2). Expressed differently, 
escaping the retinal journey is an escape from an ocular or representational approach 
into what Nigel Thrift has called the non-representational (Thrift, 2007). This impacts the 
processes of video shooting and editing in particular: at the same time as being image- 
based, the desire is to construct, edit and choreograph the images and media in a non- 
representational way. We are only just beginning to feel our way into using video-non- 
representationally. Some might say this process is doomed because images are always 
representational and that we would be better off simply using sound. Instead of migrating 
to sound, however, we work with resonance. Our non-representationality comes out of 
an approach to trusting the emergent moment of improvisation rather than framing the 
image. This invites an entirely different series of 'bindings and unbindings, becomings 
and unbecoming' (Gregg and Seigworth, 2010: 4). In other words, we try to permit one 
movement — of camera or body — to lead to another according to a different logic or intuition 
from most video practices. Following Thrift in his detailed writings on movement, or 'the 
geography of what happens,' non-representationality also rests on an oblique capacity to 
see something that is not there, or is not yet there (Thrift citing Castoriadis, 2008: 2 and 
111). We suggest that AffeXity offers an additional take on non-representationality, found 
in the complexity of an assemblage: the piece (call it a locative choreography or an AR 
performance) only exists through a combination of actions, perceptions and technological 
affordances in any given moment. Even if we wanted to control the representation more 
carefully the many components would pull apart our intentions. 



JG: I shoot. I remain calm and breathe. I shoot from my centre as if in a contact 
improvisation with the dancer and the affective gesture. I try not to direct too much, 
but rather express an allowance to the dancer, an open space in which to explore 
affect. I can sense it immediately when it is there. All the wires, plastic, glass and 
metal of the technology melts away. 



The shoot becomes very subjective and I am patient. I wait if the affect leaves the 
dancer or the resonance is not there. I try again to find a somatic connection with 
what is being captured by the lens. My eye, the lens and my body's centre always 
trying to connect with the affective resonances before me. My legs are the tripod. My 
lens is my eye. My centre is alert and all three are connecting. 
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This connection is a space that is very particular, a condensed vision of what I really 
see before me, that is the dancer in the environment. I have to 'zone in' on the 
resonances and 'zone out' the environment trying to capture the same intensities and 
resonances the dancer is sensing. I use my screendance and choreographic techne 
subconsciously, allowing it to guide these short moments of intensities. 

These subjectivities were never discussed but rather moved out in affective gestures 
and movements, leaving trails, suggestions and a semiotics of ineffability that can 
only be traced in affective gestures. 

I try to remember the Dogme principle of: 'what you see is what you get.' I now add: 
'what you feel is what you get' [10] The screendance genre becomes the perfect 
medium to capture these states of affective poesis. 



http.V/www.youtube. com/watch ?v=umlCMJ7Numg&feature=youtu.be 
AffeXity Phase 01 #1 Carlsberg 

(The online version of this paper includes embedded video as cited above and is published at 
http://twentyone.fibreculturejournal.org/fcj-150-affexity-performing-affect-with-augmented- 
reality/ -ed) 



Editing Process: Rhythmic Attunements 



Technologies provide constraints and affordances — all of us know this — but technologies 
that are in development provide an entirely new mix of constraints. As discussed above, 
Argon is a rapidly developing augmented reality browser that now supports video, but it 
is still rough and the devising period discussed in this paper coincided with very limited 
video functionality. Our visual aesthetic was layered, fleeting, ethereal but also stark and 
geometric. Being able to play with layering and opacity are of extreme importance to us. 
This corresponds with a certain aesthetic in Screen Dance that Ginslov is known for, but 
also reflects our emerging affective sensibility: for AffeXity it is important for bodies to 
be multiple and shimmering at the same time as merging with the built environment. Our 
affective sensibility far outstrips what the browser can support. Our artistic research process 
consists in moving and waiting at the same time. 
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Multi layered edit of Wubkje Kuindersmaa improvising in Copenhagen. Photo Jeannette 



JG: There is no shot list, no script and no narrative, no psychological development of 
a character in a location and linear narrative. In the edit process one uses a montage 
structure. One intuits and trusts the choices, the pace, rhythm, timing and cutting 
all connecting to the score or affective gestures and states of being expressed and 
captured. My job is to amplify that, cutting out all other extraneous information. I 
try to achieve this without too much intellectualization. I try to amplify affect and 
movement through the edits and overlays. If it is present it is there. It was a good 
take. Both camera and dancer worked in harmony. Sometimes if there was a good 
connection during the shoot, the video practically edits itself. There are no special 
effects required. 

However sometimes the intellectual processes need to surface in the edit. I need to 
shape the affect. I need to amplify the affect. I use layers and shots of the dancer 
in a moving lift, dancing in a stairwell and outside against a building. I layer these 
takes and make the moving images transparent. I amplify the dancer's face, her 
looking sublimely upwards, her hands trying to reach for spaces and moments that 
are escaping. She seems ethereal, in a dream state. The edit needs to reveal this in 
a non-linear montage, as if she were in a loop, in a lift forever reaching and moving 
sky ward. She is in the lift, the building, the stairwell, the area outside - the lift, the 
building, the stairwell and area outside are in her. They merge. 
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Directing Movement: Dislocations and Orientations 



Here we return once more to the limitations of working with an AR application that is in 
the process of being developed. Ginslov and I wanted the video to be a layer over the 
display of the mobile device, and for the edges of the frame to be incremental or soft, 
but were presented with postage stamp images. [11] In order to get past the problems 
with inserting video into Argon, the engineers requested that we provide them with video 
footage that was shot in green screen, with the background extracted. Suddenly we found 
ourselves working in a way we did not want to work — this project was all about movement 
embedded in the city not about shooting movement in the dislocated and decontextualised 
environment of a studio with green walls and floor. Green screening also did not coincide 
with our affective visual aesthetic because it tends toward the insertion of a clear edged 
'sprite' into the environment. Despite this, the strength of a good artistic-technological 
collaboration is the willingness to modify practices in the interests of development, so 
we set up a green screen in a studio and worked with dancer Niya Lulcheva. This posed 
unexpected challenges from the perspective of choreographic direction: how is it possible 
to direct a dancer into an affective state, to initiate flows of affective exchange that result 
in movement improvisation when you are standing behind a camera and she is standing on 
green cloth in an extremely cold room with cement walls and floor? 




Green screen experiments with Niya Lulcheva. Photo Jeannette Ginslov. 



(The online verion of this paper includes embedded video which is available at the URLs cited 
below- ed.) 

http://www.youtube.com/watch?v=oH6wj-GghcY&feature=youtu.be 
(green screen no chroma key) 

http://www.youtube.com/watch?v=61V8xu3Pozk&feature=youtu.be 
(green screen after chroma key) 

http://www.youtube.com/watch?v=0hNKga4d8iw&feature=relmfu 
(first mock up of possible AR effect. Note that this was only a test.) 



fibreculturejournal.org FCJ-150 87 



FCJ-150 AffeXity: Performing Affect with Augmented Reality 



JG: The merging with the environment, spaces or buildings one can imagine oneself 
occupying became the central and most evocative means to find the scores within 
which Niya had to work. She was not in a location but working in a studio in front of 
a Green Screen. We tasked her to dance as if in a space, a location, a building. 

At first she performed and danced from an exteriorized perspective. Her dance was 
large, projected and danced as if performing on a stage. We needed to guide her into 
inferiority, a subjective presence, in order to get the affect we were looking for. 

The interiorisation of the space she was imagining herself to be in only became 
obvious and affective when we asked her to blend into the building: not just move in 
the passageways, the negative spaces, the passageways that we normally occupy 
and move around in. We asked her rather move in its echoes, its mortar, its bricks 
and in its cement. 

With that her focus became inward, her dance became affective gestures, traces of 
memory and echoes of her internal vision. She was "in the zone" in the "vibe" of the 
place. She no longer had to project outwards, but allowed herself to transpire in the 
journey, her gestures became the echoes, the mortar became liquid and the building 
she was in, was in her. 

She had connected and I had shivers watching her do this. The affect had been 
passed into me. Her internal journey travelled into me via the wires, metal and 
plastic of the techne and technical. The affective had transcended the technical, the 
machinic and become poetic, visceral and liquid. 

This inventory of shimmers charts the nuances and changes in practice that make up the 
early processes of AffeXity. The metaphor of shimmer lends itself easily to speaking from the 
fleeting, never fully defined, relational moment of practice, but I want to call attention to the 
dynamic of the shimmer as a tiny moment of rupture. This moment of fissure produces an 
inkling that things might be different, and out of it a subtle change is enacted. A tiny change 
like this, whether it is in moving differently, using a video camera in an unconventional 
way or managing to coax a dancer into an improvisation that yields unexpected qualities, 
is an example of the shimmer of performance. Changes or shimmers point to a state of 
emergence, rather than seamless, circular repetition. 
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Affective Sensibility— A Reflexive Methodological Practice 



Here I will sketch the development of a particular understanding of performance that comes 
directly from the performance of affect in AffeXity [12]. I rely on the following stepping 
stones in my path: first the Austinian performative speech act, next the performative 
potential of description, followed by the performative emergence of non-representational 
actions. The result is a rough triangulation of performance across bodily movement, 
emergence and shimmering. The bodily movement keeps performance grounded in 
corporeality, but this is already a transformed corporeality in the mode of Spinozan bodies — 
of speeds and capacity to affect and be affected — that is also the mode of our contemporary 
urban bodies. The emergence is the crucial constructive or generative aspect of performance. 
The shimmering plays an unusual role, drawing attention to the fissures or uncertainties of 
affect out of which something new can arise. Perhaps this understanding of performance, 
which is also a practice of performance, is only really relevant to the performing of affect 
and will provide only a fragment to the more general consideration of performance in 
interaction design and mobile media. Even if this is so, it will be consistent with the attention 
to affect occurring through 'microscopic fragments', the 'filings of affect' (Barthes, 2005:101). 



The Performative Potential of Description 



Austinian performativity is about how language constructs or affects reality rather than 
merely describing it, and it was influential to the development of gender studies and queer 
studies in the 1990s. It is not directly about bodies or movement, but embedded in it is 
what we might now call emergence, or a generative potential — an ontological dimension of 
bringing something into being that was not there previously. On an intuitive level, I believe 
this is why so many designers are attracted to the notion of performance, combined with 
the ease of integrating the social, critical and ludic into performative practices and concepts. 
Sedgwick identifies several features in Austin's explicit performative utterances, examples 
of which include: 'I promise...' 'I dare you...' 'I apologise' 'I give up...' 'I forgive...'. They are in 
the first person singular; they are present tense; they are indicative and active; the verb in 
each one names the act (Sedgwick, 2003: 3). I would also call attention to a certain affective 
intensity that accompanies the verb. 



Is the performative lost when assertion gives way to description? Sedgwick makes 
the provocative claim that the performative, or productive, aspect of language is most 
revealing when the language is closest to claiming a simply descriptive relation to reality 
(Sedgwick, 2003: 5). 
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The provocation lies in the generally held assumption that description is a weakly mimetic 
form of language, offering little scope for fissures out of which anything new or productive 
can emerge. But this is not so. As someone who has, for many years, explored the 
convergence between thick description and dance in a phenomenological vein, I see how 
the performativity of the linguistic description works hand in hand with the performativity of 
movement. Both create openings, both are actual shifts to new grounds for thinking, acting 
and being. Linguistic descriptions of lived experience and movement improvisation — which 
can sometimes read like fragments, aphorisms or even poetry — do not point to the possibility 
of something new; they are that crystallization or transformation of something that was 
previously virtual into being. Affective description can move us to a different state of being. 
Witness your own reaction when I say 'I breathe...', 'It shimmers...' Language operates on a 
somatic level. While an elaboration of somatics is beyond the scope of this short section on 
performance, it is enough for now to say that somatics shares the field with affect. This is 
illustrated in Sedgwick and Frank's description of the experience of reading Tomkins. 



a potentially terrifying or terrified idea or image is taken up and held for as many 
paragraphs as are necessary to "burn out the fear response," then for as many more 
until that idea or image can recur in the text without initially evoking terror. Phrases, 
sentences, sometimes whole paragraphs repeat; pages are taken up with sentences 
syntactically resembling one another (epistemically modal non-factive utterances of 
the form "It is possible that...," "If ... may "Whether because..."), sentences not 
exemplifying a general principle but sampling — listing the possible. This rich claustral 
writing nurtures, pacifies, replenishes, then sets the idea in motion again. Bambi isn't 
the only terrified wild thing in this picture. (Sedgwick and Frank, 1995: 95). 



The Performative Emergence of Non-Representational Actions 



Enquiring into the quality of emergence in performance, it is also useful to explore 
Thrift's extensive writing about dance, performance and movement in the context of non- 
representationality. He finds in Deleuze a sense of 'the speech act radicalized, made into 
a tool of maximum modulation and push through which new modes of existence can be 
glimpsed, even actualized' (Thrift, 2008: 132). With the practices that make up AffeXity, 
performance as emergence occurs out of the fissures in habit and codified behaviour. This is 
the shimmer, 'the shimmering field of the body insofar as it changes, goes through changes' 
(Barthes, 2005: 73). Performance is a play between the escape and re-containment of 
movement, perception and affect. In that tiny gap there is the opening for the unexpected. 
There is risk, a risk that the performance may fail (Thrift 2008: 137), or that nothing new may 
come out of it. Emergence is an openness, not a guarantee of results. It is 'a shimmer of 
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forces' (ibid, 54) that may or may not produce. A shimmer is distinct from a judder, a shiver 
or even a ripple. It is not just a kinetic oscillating pattern, but a particular combination of 
kinaesthetic, perceptual, imaginative and affective that has a quality of unpredictability or 
magic. Like performance. 



Performance in AffeXity has an important dual role: it is a way of generating content and 
also a way of reflecting upon theories around affect. It is very close to improvisation, and 
some may read the emerging qualities as being more appropriately located in improvisation, 
but the context and many layers of AffeXity necessitate the slightly wider lens of 
performance. These many layers are also practices in themselves: dance, software design, 
social participation, and media production combined with the production of events. The 
affective turn in each of these practices does not map onto the so-called affective turn in 
theory. In short, the affective turn as it is applied to theory runs the risk of implying that prior 
to the turn materiality was lacking (Paasonen 2010). The affective turn in AffeXity involves 
subtle transformations in movement improvisation, video capture and editing, letting them 
be more liminal and less consciously directed. We have not fully formulated our approach, 
except to know already that it can never be a formula. It is therefore with some hesitation 
that I have approached the topic of method or methodology. Yet it is important at the 
moment in practice-based research in the arts and design to pose methodological questions 
and to craft new, hybrid or specialised methodological approaches that are rigorous and 
radical yet respectful of tradition, practices and materiality. [13] Equally it is clear that a 
dogmatic approach to methods is a limitation: the goal cannot be to produce a set of steps 
and to imply that good research is produced simply by following them. With that in mind, 
what I offer is more of a reflexive methodological practice, than a fully established method. I 
call it affective sensibility. 



Shimmer versus Free Fall 



There is a sort of circularity, or reflexivity to the implementation of affective sensibility: it 
is based on improvisation that comes from affect at the same time as seeking to reflect 
critically on affect. This approach invites the observation that encounters with theories 
of affect feel like a 'conceptual free fall,' possibly because 'affect emerges out of muddy, 
unmediated relatedness,' (Gregg and Seigworth, 2010: 4) of processes already underway 
or simply because it finds its ground in movement rather than stasis (Massumi, 2002). 
With AffeXity, we improvise in order to call attention to the shimmer, the changeability of 
bodies and motion in urban spaces. Affective sensibility is similar to what Barthes calls the 
'hyperconsiciousness of the affective minimal' (Barthes, 2005: 101) but it is not simply a 
cognitive state as it emerges and flows through the body in motion. The shimmer is easily 
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taken to be multiple fragmented, a series of micro changes, or mini-ruptures, but it is also an 
antidote to a sense of unrestricted or homogeneous free fall. For small ripples also provide 
toe-holds or tensions resulting in changes of direction, pauses for reflection or impetus for 
further movement. 



As a practice embedded in the moment of affect, affective sensibility is concerned with 
both philosophical reflection and the development of artistic work. It is a complexified 
approach to first person experience, because corporeal experience is given voice, but the 
locus of experience is dispersed and decentered thereby transforming first person narratives 
into narratives from affective bodies. These bodies are composed and decomposed by the 
technologies, the environment, the corporealities and the intensities that traverse them. As 
a methodological approach it can only be understood dynamically. Affective sensibility, as 
an approach to knowledge construction and research processes, recognizes that the artist- 
designer-researcher is not always in an external position to analyse affect — and sometimes 
the best way to understand its flux is from an immanent position: from the midst of and 
emerging out of. 



'Who does not want to believe we live in a world ceaselessly recomposing itself?' (Gregg 
and Seigworth, 2010: 13). Choreography is about composing actions, bodies, affects. Social 
choreographies are about recomposing them, mediating them with our lives and our many 
devices. In dark and confusing times there is a degree of optimism to be found in the 
theoretical stance that opens up scope for composing, decomposing, recomposing urban 
spaces. Yet even more than optimism, there is breathing space created in actually doing it. 
The performance of affect is an escape from paralysis, inertia or the one-sided position of 
being affected. All of have inventories of shimmers based on the practices and navigation of 
daily life; applying the awareness of affective sensibility might allow us to notice them. Then 
we have to decide what to do about them: are these the inventories we necessarily want to 
accumulate and carry around with us? In a way, this paper is less about the What Is of affect 
than it is about the How To of affect, as we grapple with it in the midst of artistic and social 
processes. 



Biographical Note 



Susan Kozel is a choreographer and philosopher working at the convergence between dance 
and a wide range of sensing and mobile technologies. She is a professor of new media with 
the Medea Collaborative Media Initiative and the School of Arts and Communication at 
Malmo University in Sweden. 
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Notes 



[1] A project that integrates arts and technologies on this scale is necessarily a collaboration 
of researchers from several domains: Jeannette Ginslov shoots and edits the images based 
on her many years as a creator of screen dance; thus far two dancers skilled in improvisation 
have worked with us, Wubkje Kuindersmaa and Niya Lulcheva; Jay David Bolter of the 
Augmented Environments Lab at Georgia Tech in the USA provides critical and technical 
expertise and is a member of the team leading the development of the augmented reality 
browser, Argon; Timo Engelhardt has experience developing media for apple devices; Maria 
Engberg of Bleking Institute of Technology in Sweden and the Augmented Environments Lab, 
Georgia Tech, is actively involved in projects using Argon for cultural heritage projects; and 
Karolina Rosenquist of Medea, Malmo University is a specialist in innovative approaches to 
audience development. I take on the hard-to-define role of artistic director by working on a 
conceptual level but also in a performative mode so I can experience the affective exchanges 
of the project in as many ways as possible. This artistic research project is located at the 
Medea Collaborative Media Initiative at Malmo University in Sweden. Information on the 
project can be found at http://medea.mah.se and www.affexity.org. 



[2] Related projects intending to create networked dance improvisation through Twitter 
includes IntuiTweet (see Kozel, 2010b, and Kozel, forthcoming 2013), Alone or Not ( http:// 
www.aloneornot.org) . 



[3] From Gilles Deleuze's book on Spinoza: 'Every reader of Spinoza knows that for him 
bodies and minds are not substances or subjects, but modes. It is not enough, however, 
merely to think this theoretically. For, concretely, a mode is a complex relation of speed and 
slowness, in the body but also in thought, and it is a capacity for affecting or being affected, 
pertaining to the body or to thought. Concretely, if you define bodies and thoughts as 
capacities for affecting and being affected, many things change. You will define an animal, 
or a human being, not by its form, its organs, and its functions, and not as a subject either; 
you will define it by the affects of which it is capable. Affective capacity, with a maximum 
threshold and a minimum threshold, is a constant notion in Spinoza' (Deleuze, 1988: 124). 



[4] It is also more than the Cartesian corporeal substance of res extensa which was defined 
in relation to res cogitans (mental substance) and god. 



[5] There is more to be said on the topic of intensities, but for now the scope for theories of 
intensity to take discussions of media beyond mere use-value or capture — 'utilizations or 
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captures' (Deleuze, 1988, 126) — had to be stated, even if briefly. For further discussions of 
intensity see (Massumi, 2002), (Bertelsen and Murphie, 2010) and (Guattari, 1995). 



[6] Donna Haraway in her seminal 1988 essay on situated knowledges called the task of 
overcoming pernicious dualities, such as that between matter and mind, both necessary 
and contradictory. Designers, philosophers and artists are still coming to grips with many 
necessary contradictions and contingencies at the present moment. It is useful when 
confronted by dualistic reactions or choices to argue for both/and as 'a necessary multiple 
desire' (Haraway 1988, 579). In a related move, Roland Barthes locates his Neutral in 'both 
at once' or 'at the same time' and calls it a 'structural U-turn' (Barthes, 2005: 190), and 
Quentin Meillassoux's After Finitude addresses the necessity of contingency (Meillassoux, 
2009). 



[7] More technical specifications on Argon can be found here http://argon.gatech.edu/ . It is 
freely available through the Apple App Store. 



[8] See Contact Quarterly's site for material on contact improvisation: http://www. 
contactimprov.net/about.html and Kozel 2012 for a discussion of improvisation and social 
aesthetics. 



[9] These words are from Jeannette Ginslov's Medea Talk in May 2012, http://medea.mah. 
se/2012/05/medea-talks-presents-jeannette-ginslov/ 



[10] Dogme refers to the approach popularised by the Dogme 95 collective of which Danish 
film director Lars von Trier was a member. 



[11] Currently Argon only runs on Apple iPhone and iPad, but plans are for it to be released 
for more general use on smartphones and tablets. 



[12] This differs from my earlier situating of performance as essentially a hyper-reflexive 
action: if one sees something as a performance, then it is a performance. This was a reading 
of Schechner through Merleau-Ponty's relation of reversibility (Kozel, 2007). 
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[13] Affective Sensibility, as a methodological approach, exists alongside two others: 
Embodied Imagination (Hansen and Kozel, 2007) and Intuitive Improvisation (Kozel, 2010b). 
These 'sister methods' attempt the following: to open up approaches to practice that reside 
in a convergence between theory and practice; to place particular emphasis on bodily or 
tacit knowledge; and to call attention to a diversifying of methodological approaches in 
academia. 
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Introduction: ordering affect and the question of labour 



Recent developments in the workplace have seen the intensification of methods to elicit and 
capture value within and across the affective encounter, notably through the introduction 
of technologies to measure the production of emotion by service workers. One of the most 
compelling examples of such methods is the 'smile-scan' - a technology developed by Japanese 
company OMRON to read and measure the intensity of facial expression in the workplace. 
Through an analysis of the use of OMRON's OKAO Vision smile scans in workplaces, this 
paper seeks to understand the insertion of particular affective technologies into the technical 
composition of capital and their role in the ordering of affect. 



In the post-Fordist condition, the role of affect has emerged as a central point of contestation, 
since the capacity to produce relationships and feeling is pivotal to the work of service, 
communicational and relational industries. Autonomist Marxists [1] in particular have pointed 
to the importance of affective labour in contemporary forms of production and circulation. 
Surprisingly, however, these theorists have paid very little attention to the new technological 
infrastructures that shape and modulate the production of affect in the workplace. As a 
consequence, their understanding of the affective conditions of post-Fordist labour neglects 
to explore the ways in which the technological composition of capital intersects with and 
informs the question of class composition. This paper revisits and complicates the insights 
of autonomist Marxism by examining the technological nexus which organises and compels 
affective labour as a force in the production and circulation of value. 
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The critique of the technological nexus and composition of affective labour will explore 
the inclusion of affect within processes of HCI on the one hand, and the modulation and 
ordering of the affects of the user, on the other. The 'turn' to affect in computing and HCI 
design has opened up a vast field of research and potential applications (see for example 
the collection Tao and Tan, 2009). This paper focuses on just one of these applications 
- namely emotion recognition technologies that seek to subordinate user affect to the 
imperatives of capitalist valorisation (productivity, rapid circulation and accumulation). In so 
doing I aim to identify how, within HCI, certain technological assemblages come to compel 
the body of the worker within a particular ordering of affect. The ordering of affect occurs 
both in the process and as the effect of the use of particular affective HCI technologies in 
the workplace. In pursuing this line of inquiry into the ordering of affect, we will be better 
placed to understand contemporary elements of the technical composition of affective 
labour as expressed in affective HCI. The use of OKAO Vision as a technology of work 
perfectly captures the tensions inherent in the relationship between user and computer, 
affect, emotion and technology. The smile-scan is a technology for the ordering of affect par 
excellence. 



Eva Hudlicka (2003: 3-6) provides a useful overview of the potential applications of HCI 
technologies focused upon user affect. Within a larger taxonomy of applications, she 
identifies one phylum of technologies that aim to sense, recognize and modulate user 
affect. In Hudlicka's words, such technologies might be deployed, amongst other possible 
uses, for 'maintaining a particular user state for a particular task' and for inducing a 
particular affective state in the user (5; emphasis in original). Both of these applications are 
characteristic of the use of smile scans in service work, in particular, and the monitoring 
of affective labour, more generally. Moreover, both of these applications point to the 
sometimes very fine line and indeed the tension that exists between affect and emotion. 
This tension is developed below, but briefly now, affect here is understood as a relationship 
of varying degrees of intensity between bodies, and the impact of this relationship upon 
the bodies involved. However emotion is taken to mean the emotional quality or disposition 
of an individual at a given point in time. Emotion relates purely to the individual. The 
examination throughout this paper looks to the position of affect within HCI used as a force 
upon labour, and which orders the affects of the user as a technology of work. 



Paying attention to the uses of HCI in the contemporary labour process, this paper will argue 
that technologies such as the smile scan form a point of refraction that articulates affect in 
a similar way to which a prism impacts upon light waves, shifting their flow, direction and 
speed. Although refraction does not change the substance of affect, it does alter the rhythm 
of movement and the perception of the flow. In this respect, we can engage the problem 
of affective labour and technology in the Spinozist terms of movement and rest, speed and 
slowness. Tension emerges as affect qua activity and labour moves through different spaces, 
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constituted by different logics and codes: value's organisation of labour determines a degree 
of movement or rest of a body and affect particular to the imperatives of valorisation. Here 
the deployment of a technological infrastructure to induce and maintain an affective state in 
a worker, mobilised within the logic of valorisation, forms the prism that compels and bends 
the affective encounter. 



Posing compositionist problems 



The perspective of compositionist analysis first emerged within the Operaismo movements 
in Italy during the 1960s, and has continued to develop in subsequent decades. The journals 
Classe Operaia (Working Class) and Quaderni Rossi (Red Notebooks), circulating the work 
of authors such as Raniero Panzieri, Romano Alquati, Mario Tronti, Antonio Negri and 
Sergio Bologna, first introduced the practice of compositionist analysis into a renewal of 
communist thought and engagement. Feminist theorists working from a compositionist 
perspective, such as Maria Rosa Dalla Costa and Silvia Federici, and groups such as Lotta 
Femminista (Feminist Struggle) radicalised the scope of the method, insisting on the critique 
of the various relations of reproduction and unwaged work as sites of antagonism (Wright, 
2002; Red Notes, 1978; Cunninghame; Bargallo & Federici, 2012). Since the early period of 
compositionist thought in the 1960s and 1970s, the method has continued to be used as 
means to problematize the organization of the reproduction of capital. 



Compositionist analysis comprises two related categories: technical and political 
composition. The general category of class composition refers to the technical organisation 
of (re)production of capitalism and the political forms of organisation that emerge both 
within and against this arrangement. The concept of technical composition takes on a 
specific dimension of meaning in this school of thought [2]. On the simplest level, technical 
composition is an expression of the structural organisation of labour-power in the production 
process, and the conditions of the reproduction of labour-power. Perhaps put otherwise, 
technical composition expresses the technological conditions, or lack thereof, that determine 
the characteristics of the labour process of a given area of work. It is also an engagement 
with the myriad features of what is required to perform specific tasks 'on the job' in a given 
area of work. The technical composition of class comes to assume, or at least inform, the 
posing of a political dimension in so far as the organisation of capitalism involves class 
antagonism. [3] Finally, technical composition articulates specific forms of the labour 
processes of respective types of work, placing the development of such processes within the 
cycles of class struggle. 
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In recent years, there have been gestures toward a renewed analysis of affective labour 
through the lens of compositionism. For example, Hardt and Negri revisit the question of 
the technical composition of class in Commonwealth (2009). Hardt and Negri's analysis 
identifies in the contemporary technical composition of class three trends: firstly the 
hegemony of immaterial production, secondly the so-called feminisation of labour, and 
finally the migratory movements of labour. According to Hardt and Negri, the three trends 
of the 'technical composition of labour.. .all are engaged in the production of common forms 
of wealth, such as knowledges, information, images, affects and social relationships, which 
are subsequently expropriated by capital to generate surplus value' (2009: 139). However, 
their examination is very general, and says little of consequence for understanding a 
contemporary technical composition of class, at the micro and daily level. For example, 
although the three tendencies Hardt and Negri point to do resonate and express in broad 
terms elements of the configuration of contemporary class composition, they cannot explain 
the ongoing encounters that mobilise the production of knowledge, affects and so on within 
the framework of capital's organisation of work. As a result, it is not possible on the limited 
terms of the 'three tendencies' to grasp the details and microelements of antagonism that 
occur in the everyday conditions of immaterial production. The perspective offered here 
by Hardt and Negri does not draw out the complex and dynamic relations that a detailed 
engagement with the terms of technical and political composition of class can provide. 
This article moves towards addressing this problem, through a critique of the relationship 
between affective HCI and labour in the workplace. 



The contribution and limitations of the theory of affective 
labour in the critique of contemporary capitalism 



The conceptual terrains upon which the theory of affective labour is developed are wide. 
Theorists of affective labour draw together Spinozist and Deleuzian concepts of affect and 
a Marxian critique of capital to interrogate the process of value creation and exploitation 
in post-Fordist capitalism (see Spinoza, 1996, Deleuze, 1988 & 2005, Negri, 1999, Casarino 
& Negri, 2008, & Hardt & Negri, 2000, 2004 & 2009). Whilst the specific engagement with 
the works of philosophers such as Spinoza and Deleuze constitutes a key conceptual field 
for the development of the theory, feminist critiques of political economy, emotional labour 
and the body have also been pivotal. [4] Drawing from feminist critiques of emotional labour 
and 'reproductive' work, autonomist Marxists have developed the theory of affective labour 
within the broader conception of immaterial production. 



The work of theorists such as Michael Hardt (1999), Antonio Negri (1999), and Franco Berardi 
(2009a), represents one attempt to rethink labour and work in a contemporary context. 
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For these theorists the condition of post-Fordism is in fundamental ways characterised 
by the emergence of immaterial production. For Hardt and Negri, immaterial production 
sees 'images, information, knowledge, affects, codes, and social relationships. ..coming to 
outweigh material commodities in the capitalist valorisation process' (2009: 132). [5] The 
emergence of immaterial production is marked by an increase in the amount of labour 
carried out in service industries generally. Perhaps more importantly, with immaterial 
production the work of communication, cultivating relationship and producing affect comes 
to constitute primary points in the organization of accumulation in contemporary capitalism. 
In other words, the resources, raw materials and labour used in immaterial production are 
directly affect, language and knowledge. This is what is meant when it is said that today, 
fixed capital and the general intellect are increasingly located not in machinery but in the 
bodies and brains of workers. 



In a somewhat different register, emphasizing the pervasiveness of communication to 
contemporary production Paolo Virno has argued that the 'communication industry.. .plays 
the role of industry of the means of production', in so far as techniques of communication 
and relation become means of production in post-Fordism (2004: 61, emphasis in original). 
The passage to post-Fordism, which brings us to the present conjuncture, complicates the 
relationship between time and labour, and the techniques that articulate this relationship. 
Christian Marazzi (2008: 41) brings this complexity to the fore, emphasizing the centrality of 
language and communicative-relational action in post-Fordist production. Marazzi argues 
that 'the chain of production has... become a linguistic chain, a semantic connection, in 
which communication, the transmission of information has become both a raw material 
and instrument of work' (50, emphasis in original). Here, language and communication 
become the stuff of production and the means of wielding it; production sees the linking 
of material and execution of labour in language and communication. However, it is also 
necessary to locate affect and emotion specifically within the post-Fordist economy. 
Berardi's articulation of semiocapitalism maintains that 'productive life is overloaded with 
symbols that not only have an operational value, but also an affective, emotional, imperative 
and dissuasive one' (2009(b): 107). In the passage to post-Fordism, the labour of producing 
affects, communication, knowledge, the creation and maintenance of relationships and the 
cultivation of attention emerge as key economic terrains. 



On a higher level of abstraction the theory of affective labour produces a number of political 
and philosophical problems. For Hardt and Negri, affective and immaterial labour do not 
produce commodities, but rather social relationships and ways of life. In other words, 
the category of labour merges with that of life, and thus becomes a singular productive 
substance. Immaterial production and affective labour therefore become biopolitical 
production: work that produces the very fabric of social life and of the common. The 
emphasis on biopolitical production marks a key element in the ontological turn in post- 



102 FCJ-151 fibreculturejournal.org 



Mark Gawne 



workerist thought. The limitation of the ontological turn in theories of affective labour 
becomes clear when we return to some of the critical resources provided by earlier examples 
of compositionist analysis within autonomist Marxist theory. 



All of the above authors have, in their own way, attempted to come to terms with the new 
forms and configurations of labour. In doing so they have identified the various forms of the 
direct relationship between the body, subjectivity and the labour processes of immaterial 
production (Eden, 2012). Although some of the primary theorists of immaterial production 
and affective labour emerged from the perspective of compositionist analysis, the theory's 
construction has dispensed with some of the key theoretical insights of the earlier period of 
research and political practice. The key limitation of the analysis of the class composition of 
immaterial production arguably lies in its flattening, or collapsing, the categories of technical 
and political composition into the singular foundation of biopolitical production, or productive 
life. [6] It is in this move, however, that the rigour of the critique of labour and work is lost. 
The perspective developed in this paper seeks to avoid the problems of this ontological turn 
through refocusing on the question of the technical composition as expressed in affective HCI. 



The complexities that emerge when we consider the technological nexus that is developing 
specifically within the factories of affect, whether these are call centres, hospitals, retail 
outlets or any other customer service floor, warrant a more nuanced consideration of 
contemporary forms of technical composition. The point of departure for such a consideration 
involves, following the insights of early compositionist critiques of class and work, the 
technical, including the technological, organization of labour. It is with this point of departure 
in mind that the analysis of this paper will approach both the question of affect within 
HCI, and also that of developing the complexity of the theory of affective labour and its 
composition. Whilst theorists of affective labour have identified the need to consider the 
problem of immaterial production, they have less to offer in critiquing how the relation of 
capital and its various technical elements have also taken up and incorporated this same 
problem of affect and labour. The critique of affective HCI in this paper thus aims to situate 
affective HCI, specifically OKAO Vision, as an element within the structural organization of 
labour-power. In doing so, it will be possible to look firstly at the specific implications of the 
use of this technology at work, and secondly, to develop some general points to be applied 
to the more broader analysis of affective labour and immaterial production. We will return to 
the discussion of composition and HCI towards the end of the paper, with an emphasis on the 
elements of technical composition. 
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Affect and emotion 



On a superficial level it appears that the various theorisations of affect have in common 
an understanding of affect as a bodily and relational force. Yet in certain disciplines and 
registers affect and emotion can be used interchangeably. It is therefore useful to be clear 
about the difference between these categories, particularly as we are attempting to come 
to terms with the process of modulating and ordering affect through HCI at work. The 
definitions used throughout this paper will follow those of Gilles Deleuze and Brian Massumi 
(Deleuze, 1988; Massumi 2002). Following Deleuze, affect is an intensity of feeling that runs 
across, between and through bodies, but which cannot be captured within an individual 
body. The sense of relation is thus never closed, and affects cannot belong to a subject but 
must be a flow of intensity between bodies expressed as passion or action. As Massumi 
suggests, emotion will necessarily emerge from an affect, but it is effectively objectified 
and possessed by an individual. Emotion becomes the individual's, the subject's expression 
of feeling. In this respect, perhaps, emotion represents the reduction of affect as open flow 
to something that is quantifiable, fixed and exchangeable. It remains to be seen, however, 
whether various affective HCI technologies necessarily carry out such a reduction, or 
whether they rather generate a unique process of order. 



In Post-cinematic Affect (2010) Steven Shaviro makes the argument that affects bear the 
same relationship to emotion as does labour to labour-power. The first in each of these 
respective relationships, affect and labour, can be described as the open intensity, and the 
non-containable force: labour is the form giving fire of human creativity; affect is the open 
intensity of relation that cannot be individualised and objectified (Marx, 1973, p361; Spinoza, 
1996). In contrast, the categories of emotion and labour-power are the objectified form of 
the creative intensities from which they emerge, and which they represent. Emotion is a 
contained, personalised moment of affect, while labour-power constitutes the captured, 
commodified expression of labour (Massumi, 2002; Marx, 1951, pp74-97). Building upon 
this distinction, we come to the following problems: if the exchange of labour-power is 
a necessary objectification of labour within the social relation of capital, it might seem 
difficult to talk of an affective labour within the valorisation process, based on the terms 
of distinction outlined so far. In a different sense, we also need to clarify the role of HCI 
technologies, in this case OKAO Vision, in the determination of affective or emotional 
relations. We will return to these problems throughout the paper. 
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HCI and the circuits of order and amplification 



Recent trajectories in HCI, drawing from work in affective computing, have sought 
to emphasise the affective state and expression of users in the design of interactive 
technologies, and to develop the capacities of machines for affective display (see Zeng et 
al., 2007; Gunes et al., 2004; & Truong, 2010). Rosalind Picard's (2000) work on affective 
computing opened up a field of enquiry which has subsequently been taken much 
further by researchers and designers in the field. The variety of innovations emerging 
from affective computing and HCI is vast and include eye tracking instruments, devices 
which interpret the emotional significance of physiological data such as temperature and 
heart rate, as well as emotion and gesture recognition technologies, to name just a few 
(see Gunes et al., 2004; Lao & Kawade, 2004; Zhao et al., 2003; Jaimes & Sebe, 2006). 
The diversity of these trajectories all indicate a general 'turn' to affect in computing and 
interactive technologies. 



HCI, understood as a broad field of research and development, focuses on the improvement 
of the relationship between users and computers. However, the engagement with affect 
in recent decades has contributed much to expanding this field. Speaking generally, prior 
to the affective turn, most research in HCI was concerned with the logical and calculative 
powers of the computer, while studies in human-machine interaction focused upon the 
adaptability of the user to the computer. [7] Here, it was assumed that it is easier to get a 
person to adapt to the rigidity of a machine/computer than to get the computer to learn 
and adapt to the user (Raskin; 2000). More recently, as Noam Tractinsky et al. (2000) have 
demonstrated, researchers have uncovered empirical correlations between designs that 
address the affective dimension of an interface and its perceived usability by humans, 
constituting evidence that challenges the previous wisdom of function over form. The 
increasingly sophisticated engagements with affect are productively complicating the 
frameworks through which computer and interface design and development is thought. 
Not surprisingly, in the relatively short period of time in which affective computing and 
technologies have emerged as a field worthy of serious consideration (Hudlicka, 2003; 
Sengers et al, 2002; Boehner et al. 2005; Partala and Surakka, 2004), numerous divergences 
have emerged. These divergences reflect the difficulties inherent in the very definition of 
the concept of affect and cognate understandings of interaction. 



Some of the initial engagements with affective computing approached the problem of the 
direct relationship between an individual user and his or her computer. Picard (2000:1) 
defined affective computing in terms of computing that either relates to, influences or 
arises from emotions. This implies that the computer should be able to recognise emotion 
in the user, recognise and respond to this emotion, and in other cases, that the computer 
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itself possesses emotion of its own. In a paper linking affective computing and HCI, Picard 
(1999), identified four areas of development: reducing user frustration, communication of 
user emotion, developing the means to handle affective information, and finally development 
of social-emotional skills. In each case, the aim is to improve the fluidity of the relationship 
between an individual user and a computer through a direct attention to the dimension of 
emotion in computing. 



Picard's conceptualization of a one-to-one affective relationship between user and computer 
reveals a number of practical and theoretical limitations. In the first instance the slippage 
in terms from affect to emotion opens the way for a conception of affect as a quantifiable 
substance, which can be measured, interpreted, learnt and directed. Following the argument 
of Boehner et al. (2005), what we have here is an example of an informational model of 
emotion. In their words, 'emotion, in the informational model, is a dual of cognition, but 
it is nonetheless the same sort of phenomenon - an internal, individual and delineable 
phenomenon, which acts in concert with and in the context of traditional cognitive 
behaviour' (p59). Thus the potential for opening a deeper affective engagement within the 
confines of this informational model is limited by this reduction of affect to a quantifiable 
unit. This, in fact, has reproduced some of the very problems that advocates of affective 
computing had identified and critiqued in the cognitivist approaches to artificial intelligence. 
A further limitation can be identified in the personal nature of the relationship between 
user and computer. The construction of the concept of affect, emotion and relationship here 
effectively reduces these terms to fixed possessions held either by the user or the computer. 
The affective relation here is an interchange of fixed units of emotion, determined through 
the reading of indicators on the body, or in the expression of the computer. Affect itself is 
defined as a fixed possession, or inherent feature, of a person, precluding any understanding 
of its social, cultural or political dimensions (Sengers et al., 2002). 



To counter this notion of affect as information Boehner et al. (2005) develop an interactional 
model as a way of constructing a more complex framework through which to approach 
the question of affect and emotion. Within the interactional model affect is not reduced to 
either a possession of the individual or a characteristic that can be fixed within a computer. 
Rather emotion and affect are placed in a social and cultural context, and as such are seen 
as dynamic arising from action and interaction. Within the interactional model we can see 
affect as a form of intensity produced in the relationship between the user and computer. 



The theoretical openings and limitations produced in affective HCI, such as the informational 
model, has led others to push the potential of interactive design in the direction of 
constructing environments or spaces of becoming. Jonas Fritsch (2009) draws on Massumi's 
crucial theorisations of affect to deepen the prospects for affective engagement in HCI. As 
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is well known, and pointed out above, Brian Massumi develops the Deleuzean and Spinozist 
theorisations of affect, through an array of discussions of how a body, already constituted 
by various intensities and potentials, that is, affects, is in turn affected and thus moves to 
a higher or lower capacity to act (Massumi, 2002). Affect is a pre-personal potential and 
intensity, which is irreducible to the level of the personal or individual. Affect is inherent to 
and mobilised in experience and event, which is to say affects move and reconstitute given 
bodies. To affect or be affected is thus to be within a process of becoming, to a greater 
or lesser capacity to act. Fritsch discusses these theorisations of affect in the context of a 
public interactive installation called Touched Echo (Markus Kison). The interaction with the 
installation creates an open affective engagement through the orientation of the interaction 
and the transformation of the physical space in which it is installed. What is significant in 
Fritsch's discussion is the investigation into the installation's production of an amplifying and 
expansive affective encounter. Fritsch's contribution to the field pushes to a further level the 
conception of affect within interaction design, beyond models of emotion and information. 



Based on the above discussion of tendencies within the development of affective 
technologies, it is possible to make a qualitative, yet fluid, distinction within the field. 
On the one side of this demarcation lie those innovations in the application of affective 
technologies for the amplification of affect, and on the other side lie technologies for the 
ordering of affect. The demarcation is fluid in so far as the distinction between amplification 
and ordering could shift depending on where and how a given technology is mobilised 
and functions within the materiality of social relations. Yet it is useful as it allows us to 
problematise the relationship between technology and affect, and going further to grapple 
with the tensions involved in their deployment. Technologies of amplification can be defined 
as those situations in which the orientation of their use is in the construction of an expansive 
affective engagement and environment, as discussed above in the works of theorists such as 
Fritsch (2009) and Boehner et al. (2005) Here we encounter an affective resonance opening 
out to processes of becoming. On the other hand, ordering technologies function not through 
resonance and expansion but act as a directive force upon a body's behaviour. It is to 
examples of these latter technologies that we now turn our attention. 



The original online version of this article inlcudes an embedded video: Figure 1: 'Smile 
Training for Japanese Workers', New Tang Dynasty Television, 2009. Permission to embed by 

New Tang Dynasty Television. 

That video is available at http://youtu.be/ReC86fylpJQ and the original article is published at 
http://twentyone.fibreculturejournal.org/fcj-151-the-modulation-and-ordering-of-affect-from- 
emotion-recognition-technology-to-the-critique-of-class-composition/ 



fibreculturejournal.org FCJ-151 107 



FCJ-151 The modulation and ordering of affect. 



OKAO Vision & assemblages for the ordering and modulation of 
affect [8] 



In its 2009 annual report, the company OMRON provided a brief overview of one of its most 
recent, cutting edge and 'exciting' sensor technologies. The name of this sensor technology 
is OKAO Vision, commonly called the smile-scan, and is OMRON's most advanced model 
of a facial recognition technology. The senior manager of OMRON's sensing and control 
laboratory, Masato Kawade, has stated that 'the technology has great potential for a whole 
host of applications from consumer electronics to healthcare. Imagine a camera that makes 
sure you don't miss that moment when everyone is smiling at their best. Or what about a 
'smile-checker' for people working in the service industry?' (cited in Control Engineering, 
2007). Evidently, the software itself is quite versatile, and as such there are a number of 
applications for which the facial imaging and sensor technology can be used. In applications 
designed for the private consumer these include hand held photography cameras, video 
game applications, and in personal computers for purposes of security such as biometric 
login. In other instances, facial recognition technologies have been installed in public areas, 
such as town squares or transport areas with high pedestrian traffic, for identification and 
surveillance, including the ability to identify individual faces within large crowds. However, 
of importance for the present discussion is the use of the technology for commercial 
applications within the workplace. OMRON's OKAO Vision has been installed for use in 
service and care industries, where it is used for training and on the job monitoring of the 
worker's facial expression. It is this latter deployment of OKAO Vision as a directive upon the 
worker's body that I will analyse below. 




Figure 2: Screenshot from 'Smile Training for Japanese Workers', 
New Tang Dynasty Television, 2009. 
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Facial recognition technologies function by reading, categorising and responding to a 
person's face. The basic steps of the operation include fixing an image, reading it and 
providing a feedback loop with the user. The software first fixes an image then detects 
the face within the image. The feature points of the face, eyes and mouth, for example, 
are located within the image. Once the computer has recognised and interpreted the 
configuration of the feature points of the face, it is in a position to assess the overall image. 
The process provides the foundation for the facial recognition of the user. The information 
accumulated from the individual user's face is compared against the OKAO Vision library or 
database, which in turn informs the feedback to the user, including an estimation of gender 
and age, as well as facial condition including expression, gaze, and finally a facial image 
optimisation (Lao & Kawade, 2004). 




Figure 3: Screenshot from 'Smile Training for Japanese Workers', New 
Tang Dynasty Television, 2009. Permissions provided by New Tang Dynasty 



The smile scan follows much the same series of operations, but adds layers of detail along 
the way. The scan itself locates a larger number of points on the face, such as the tips of the 
eyes and mouth, points on the cheeks. Reading the individual's face and the position of their 
features, the software then constructs a model image, a three dimensional configuration 
of the face, and compares this against a library of facial images in its database. Given the 
measurement of the face and the location and shape of the key features, the software is 
then able to provide a feedback rating and assigning a value measurement of the individual's 
smile, out of the potential result of one hundred percent. It does this through measuring 
the movement of the key features of the face as it smiles. The categorisation and response 
generally involves the assignment of a given value and emotional reading of the individual's 
face. 



fibreculturejournal.org FCJ-151 109 



FCJ-151 The modulation and ordering of affect. 




Figure 4: Screenshot from 'Smile Training for Japanese Workers', 
New Tang Dynasty Television, 2009. Permissions provided by New 
Tang Dynasty Television. 



At the time of writing, existing reports show that the OKAO Vision technology has been 
in use in various service, retail and health workplaces for the purposes of training and 
monitoring employees whilst at work. At Keihin Station, the smile scan technology is in 
use for the purposes of regulating the bodily performance of workers at the station and 
for the purposes of training. In this case, workers at the station undergo a smile scan at 
the beginning of a shift. The technology, as indicated above provides a measurement and 
ranking of the 'potential smile' of the individual, a percentage out of one hundred. The 
point of this process is to develop in the worker the most perfect smile for the individual to 
perform whilst on shift. It is reported further that they retain a print out of their smile scan 
to remind them throughout the day (Meyers, 2009). Each of these applications operates as 
a directive on the body of the worker to perform a repetitious but familiar appearance of 
intimate labour. 



There are in fact two interfaces in the process of production involving OKAO Vision. The first 
is that determined in the relationship between the user (worker) and the computer (OKAO 
Vision). On first appearance this relationship functions within a closed circuit. The worker 
engages with the computer, which interprets, measures and feeds back the information 
about the performance of the worker. However, given the fact that the use of this circuit is 
to modulate the performance of the worker in a second environment, which might be put as 
the shop floor, the closed circuit at a given point opens onto this second space. At this point, 
the body of the worker, having already been engaged, interpreted, and modulated/compelled 
by OKAO Vision, becomes itself a second interface expected to produce a desirable 
affective encounter with those other bodies it comes into contact with whilst at work. In 
the connection between the time of the worker's first encounter with OKAO Vision, and the 
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ongoing performance throughout the worker's shift, we witness the broader assemblage 
that modulates the form of affective labour carried out on the job. This larger procedure, 
consisting of the first encounter between user-computer and the opening of this circuit onto 
the second space of the shopfloor, situates OKAO Vision as a central component of the 
assemblage for the ordering and modulation of affect. 



It is useful here to locate OKAO Vision alongside another example of the use of affective 
technologies for the modulation and ordering of affect in the workplace. In a similar way to 
the smile-scan, the use of vocal emotion recognition technologies in call centres produces a 
relationship between user and computer that compels the performance of a specific affective 
labour. This involves the use of vocal emotion recognition technologies in the training of 
workers, as well as at the point of interaction. Diverted to Delhi, (Stitt, 2003), demonstrates 
the use of HCI as Indian call centre workers are trained to perform correct communication 
with clients. The outcomes of the training and monitoring consist in the pronunciation of 
words, accent training to sound either accent neutral or 'native' to the country from which 
they are calling and/or receiving calls, and finally training in vocal pitch and tone. As is the 
case with the use of OKAO Vision, the first circuit between the worker and the technology 
produces an affective performance, which the worker then repeatedly enacts in the 
encounter with other bodies when on shift. [9] 




Figure 5: Screenshot from 'Smile Training for Japanese 
Workers', New Tang Dynasty Television, 2009. Permissions 
provided by New Tang Dynasty Television. 



The implementation of OKAO Vision in the workplace functions through the measurement 
of the performance of an emotive display for the creation of an affective encounter: it 
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creates the possibility for the indexing of the efficiency of affective performance. The 
notion of efficiency in service work often takes on new meanings, as it is difficult to subject 
practices such as care or attention to the usual standards of productivity and metricity. 
However in a context in which relation, attention and emotion become key considerations 
of economic organisation, the modulation and intensification of affect is paramount. Upon 
this foundation, affective technologies are linked to the effectiveness of the worker, and 
to this degree a conception of efficiency within the workplace. The application of emotion 
recognition technologies such as facial and vocal emotion recognition is already evident in 
some service industries involving human-human affective interaction and human-machine 
affective interaction, or both (see Omron, 2009; Vora, 2010: 33-47; Records et al. 2007). 



Returning to problems of composition 



Throughout the remainder of the paper we will problematise the theory of affective labour 
in relation to OKAO Vision and the broader issue of work and technological assemblages 
for modulating affect. Having considered the terms of class composition, and of HCI and 
OKAO Vision specifically, I would like briefly to draw these threads together, returning to 
the problem of composition. It is clear that the development and innovations in affective 
computing and their application within HCI are now critical elements within computer 
design. The importance of the turn to affect is evident in the discussion throughout this 
paper. In the condition of ubiquitous computing, the importance of innovations in affective 
design of interfaces will only grow. At the same time as this tendency develops, we see 
that there is a mirrored importance of affective HCI in the dynamics and forms of technical 
composition of class today. Inasmuch as affective labour is, in many workspaces, mediated 
by the computer interface, the configuration of affect as a quantifiable substance (or 
emotion) to be modulated in the interests of securing customer satisfaction and loyalty is 
enabled by theories of HCI that focus on the instrumental relation between the user and the 
computer. The smile scan can be understood as an exemplar of this. 



In another respect, however, and returning to the conditions of post-Fordism and 
ubiquitous computing, we again see the importance of coming to terms with the 
emerging technological infrastructure oriented to organize the production and circulation 
of affect. Within the framework of immaterial production, the social composition of 
affective labour goes beyond the technological mediation of the computer to seep into 
the entire infrastructure of affective relations that constitute the post-Fordist workplace. 
From this perspective, with or without the direct mediation of such technologies as the 
'smile scan', the post-Fordist process of valorisation necessarily attempts to subsume the 
affective relations of society, and thus is intent on the ordering of affect, and perhaps its 
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transformation into emotion. The critique of the technological nexus of affect is thus 
relevant for understanding the composition of affective labour at both the immediate 
encounter with technology at work and throughout the social terrain. 



The attempt to impose a metrics and form of measure onto affective relations generates 
a refraction of affect from the perspective of the worker. The process of refraction 
operates through the modulation of the production of relationship imposed by particular 
affective HCI technologies. The attempt to impose measure, and thus transform affect 
into something quantifiable, sharpens the relationship between affective labour and 
perceptions of worker efficiency. Further, the relationship between affective labour and 
efficiency is evident in the literature and management techniques that are concerned 
with worker performance of affect and, for example, the interpretation and development 
of indicators of sales rates of workers linked to affective engagement (Byron, Terranova 
& Nowicki, 2007; Bjerg & Staunaes, 2011) [10]. Affective management techniques work 
as elements alongside affective HCI in the workplace as an apparatus of compulsion and 
refraction. As Bjerg and Staunaes point out, managerial technologies of affect are designed 
'to energise the register of affectivity', to 'concentrate on the production and formation 
of intensity' and that in this framework 'affects and affectivity are not simply by-products 
or something to be overcome, but the core matter to be managed by and through' (139). 
Whilst it may be theoretically untenable to subject affect to measure, to reduce it to 
something quantifiable, the techniques of modulation and measure demonstrate the 
will to subsume, and to order, that the process of post-Fordist valorisation mobilises. It 
is through the lens of compositionist analysis of these elements of affective labour that 
future research and critique will be best placed to understand the forms of tension and 
antagonism such a condition produces. 



Prism, control, refraction 



The OKAO Vision smile-scan, and its use in the workplace, functions as a directive upon 
the body to produce a particular affective performance. This performance in turn can be 
characterised as a modulation of affect on two levels, occurring simultaneously. Firstly, 
the scan compels the worker's body to perform affect. In other words, this is a regulation 
of labour, but not in a linear temporal sense, but rather in compelling the body to produce 
an affective intensity. The first element of the modulation of affect takes place in the 
circuit between the worker and the computer. The next element in the modulation of affect 
takes place in the second space of the shop floor, in the encounter between the worker 
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and the bodies it engages. In other words, once processed by the smile scan, the workers 
reproduce a particular form of engagement or relation with those they come into contact 
with. The labour they perform is certainly involved in the production of communication, 
relationship and affect. However, the organisation of the material constitution of this work, 
problematises, rather than confirms, the literature concerning affective labour and affective 
or emotion recognition technologies. 



The applications of the smile-scan technology function as a directive upon the body in 
a repetitious compulsion to perform affective labour. The process of affective labour is 
identified and regulated through repetition in the smile scan, in order to reproduce a 
performance of intimacy. It is through this process that affect and labour become indexed 
and ordered through the value rating and training organised through the application of 
OKAO Vision in workplaces. The deployment of OKAO Vision in the context of affective 
labour produces the homogeneity of a striated space in which work is organised. 



Spinoza theorised affect in the interaction between bodies, and the means by which they 
come to determine other bodies to different states of movement or rest, and express what 
happens to the modes of substance (Spinoza 1996; Deleuze 1988: 48). The concept of the 
technological infrastructure as a prism and point of refraction is introduced as a way to 
engage affect, labour and value whilst retaining the Spinozist foundation of affect and 
singular substance. In other words, there is no need to pose a restoration of essence of 
being, or the completely alienated subject to be restored. Rather the tension or antagonism 
is located through the prism as it compels the body and affect through the technological 
articulation of labour. This necessary movement through the OKAO Vision technology, as it 
orders the affect of the worker, constitutes the point of refraction of affect. In this movement, 
the affects of the worker's body are determined to a different degree of movement, in turn 
producing a particular affective encounter through the space of work. The worker's body and 
affects are not removed, but rather engaged, modulated and ordered in the double interface 
produced by the deployment of OKAO Vision in the workplace. In this respect the imperatives 
of value's self-valorisation determines the degree of movement, and repetition, to the 
worker's affective labour. 



The use of emotion recognition technologies in job training and at work, thus points towards 
a novel form of command in the workplace and of the performance of labour. Alongside 
this, the mobilisation of affect in management discourse and technique, specifically as 
a mechanism for intensified self-management, demonstrates a further dimension to the 
integration of analyses of affect, understood broadly, to the command of labour. Emotion 
recognition technologies and affective management techniques, in this sense, represent an 
attempt to introduce/impose a form of measure to the production/circulation of affect in 
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contemporary work arrangements, and thus to link quantifiable standards and notions of 
efficiency to the command of affective labour. 



Conclusion 



It is increasingly evident that the turn to affect is a central element in the future of computer 
design and for HCI technologies. The present innovations within HCI and design demonstrate 
this tendency. Whether these innovations are developing the informational model of affect 
and increasing the level of sophistication of these technologies, or whether designers and 
theorists are pushing beyond this model to radicalise the experience of interaction, it is 
clear that the engagement with affective design is of key importance. However, whilst the 
relationship between affect and interface design has been developed and interrogated in 
various ways, the field has not addressed to a similar degree the implications for how we 
think about labour and work in relation to affective technologies. And yet, in the post-Fordist 
condition of the expansion of labour involved in communication, emotion and affect, the 
tendency for these technologies to confront labour as a directive force or control technique 
becomes a key area of tension. This paper has sought to contribute to opening this area of 
research and critique through integrating a discussion of affective HCI with the perspective 
of compositionist analysis and the modulation and ordering of affect at work. 



Developing a compositionist approach allows us to situate affective computing and HCI, in 
this case OKAO Vision, within a specific relationship produced by the valorisation process 
and in the workplace. The discussion of OKAO Vision provides a brief insight into the circuits 
of interaction between user and computer in the workplace, and the process of modulation 
and order produced in this situation. Rather than a simple interaction between user and 
computer, in the context of the workplace, affective technologies can function as a directive 
upon the body of the worker in the first moment of interaction, which in turn produces a 
homogenous form of interaction between the worker (user) and those bodies it engages on 
the job. In this respect, the discussion of OKAO Vision, in the context of a compositionist 
perspective, opens some possible directions for developing this area of critique in 
understanding the impact of affective computing and HCI on the social and political 
dimensions of work. 



A second and minor element of this paper, again through emphasising the compositionist 
approach to analysing affective technologies, allowed us to problematise the terms 
and limitations within the theorisation of affective labour. This task was carried out 
primarily through the identification of the limitations concerning the understandings of 
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the technological organisation of affective labour, and pointing to the need for a serious 
reconsideration of the technical composition of class. These limitations, it was pointed 
out, arise primarily from the ontological turn of post-workerist elements of autonomist 
Marxist theory, evident in the collapsing of the categories of life and labour, into a singular 
productive substance. This ontological turn has meant that these theorists have marginalised 
the very material, everyday organisation of this labour. However, a return to a compositionist 
analysis, which picks up on the emerging technological nexus involved in affective and 
immaterial labour, specifically affective HCI, provides a means to address these limitations. 
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Notes 



[1] Throughout this paper I will generally use the term autonomist Marxism to refer to the 
theoretical literature that has developed the theory of immaterial production, affective 
labour and compositionist analysis. This term is used for the sake of simplicity, and because 
in English this is the most common name for what is in reality a heterogeneous constellation 
of perspectives. Although using the Italian terms of Operaismo and post-Operaismo, naming 
the earlier and later tendencies of autonomist Marxism would provide more accuracy, for the 
limits of the present paper, autonomist Marxism is sufficient, and provides greater continuity 
throughout the paper. 
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[2] In the opening of chapter 25 of Capital vol.1, Marx outlines his conception of the 
composition of capital. For Marx, these include the value composition of capital, the 
technical composition of capital, and the relation between these in the organic composition 
of capital. As is so often the case in Marx, these different categories express the material 
and a social dimension of capital. The technical composition for Marx is the material 
expression of capital in the relationship between variable and constant capital, whilst the 
social composition is expressed in the value composition. These each express, in different 
ways, the ratio between constant and variable capital, between the means of production 
and living-labour. The relationships outlined here by Marx, give an indication of the notion 
of compositionist analysis within the Autonomist thread of Marxism, however the latter 
perspective adds a twist to Marx's explanation, through a more direct engagement with the 
subjective elements of class within the framework of composition. 



[3] Throughout this paper I am focusing on some nuances that emerge in the question of 
technical composition, and simply pointing to how an engagement with this element is 
necessary within the analyses of class composition and affective labour today. Whilst this 
is a limitation on the scope of the argument and does not provide the workers' perspective, 
it is in line with the objectives of the paper. For discussions on political composition and 
its relationship to technical composition, see Sergio Bologna, 'Eight Theses on the Militant 
Historiography'. 



[4] Although feminist scholarship is often cited as a pivotal source in the development of 
the theory of immaterial production and affective labour, and indeed is in a number of 
respects, it is important to note the serious criticisms made by feminists of the immaterial 
production theses. These criticisms charge that the theory of affective labour effectively 
marginalizes, excludes or ignores the critical insights into the critique of political economy 
made by feminist scholarship at least since the 1970s. In carrying out such a marginalization, 
the specific power dynamics and hierarchical divisions in the various forms of 'reproductive' 
labour are smoothed over in the generality of the theory of affective labour. I am in 
agreement with these criticisms, although I do not address them explicitly in this paper. See 
for example, Alessandrini (2012), and Schultz, (2006). 



[5] There is significant criticism of the theory of immaterial production, particularly in such 
claims that it 'outweighs material commodities' (See Henwood, 2003). Hardt and Negri have 
counter argued that they mean immaterial production outweighs, or is hegemonic, in the 
sense that it has a qualitative effect upon other areas of production (See Multitude, 2004: 
109). This particular debate is not relevant to the concerns of the present paper, as here I 
am not analysing the empirical quantity of service work, or the hegemony of immaterial 
production, but rather looking at the material organisation of labour within service industries 
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and the forms in which affect becomes a contested terrain within this work. 



[6] Hardt and Negri claim to invert Foucault's conception of biopower to include the 
perspective of the resistance and the productivity of the multitude. They name the 
perspective produced by this inversion biopolitical production. Biopolitical production 
becomes, for Hardt and Negri, the production of the common, of knowledge, affects codes 
and so on. As such, biopolitical production is a key element in the ontological turn of post- 
workerist thought. However it is also in the centrality of biopolitical production for the post- 
Operaisti that the simplification of the technical problems of work takes place. For Hardt 
and Negri's discussion of their reading of Foucault's biopower, and their presentation of 
biopolitical production, see Hardt and Negri (2000, 2004 and 2009). 



[7] The shift in design focus to engage affect marks a significant break with previous 
emphases in computational and interaction engineering and design that effectively 
prioritised function over form. Elizabeth Wilson's recent book Affect and Artificial Intelligence 
(2010) draws attention to the importance of affect in Turing's thinking on artificial 
intelligence, uncovering an often-neglected history of affect in computing. However it 
remains the case that despite this minor thread of affect in computing, by and large, until 
very recently, the power of cognition and calculation has constituted the index against which 
intelligence is measured. 



[8] See New Tang Dynasty (2009) for use of OKAO Vision. 



[9] It is not the intention here to argue that the use of affective technologies in the workplace 
is inherently negative and restrictive. Moreover, the discussion in this paper has avoided 
engagement with the workers' perception of the use of OKAO Vision in their workplace. 
Ariel Ducey's Technologies of caring labour: from objects to affect provides an insightful 
discussion of the complexities and nuances of perception concerning the use of various 
affective technologies in the training of health care workers in the United States. Although 
Ducey does not look at HCI or indeed any forms of computerized technology, her analysis 
can be of use to us. Ducey highlights both the limitations of the use of technology in soft 
skills training, and the problems that the commercial logic within which they are deployed 
creates. Yet she also points out the potentialities the technological objects may hold for 
opening and expanding the capacities of workers to affect and be affected whilst carrying 
out care work, a form of work that 'produces society itself. The points raised by Ducey 
are important, because they point beyond the contexts of the use of emotion recognition 
technologies in call centers and the use of the smile scan. It is clear though that the use of 
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OKAO Vision in the workplace does not play a role in the production of soft-skills, as is the 
case in the discussion provided by Ducey, except in so far as being trained to smile could be 
said to count as a skill. 



[10] Bjerg and Staunaes (2011) provide an insightful analysis of the development of affective 
management techniques and the shift from reflexivity. They develop upon the work of 
Massumi and Deleuze in developing the critique of how affect as a mobilizing force is 
being deployed in the management and self-management of individuals and workers. For 
a broader discussion of the history of affect within the workplace, see Gary Latham, Work 
motivation: history, theory, research (2007). 
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Andrew: Erin, before we discuss the implications of 'Entertaining the environment' [1] with 
an artwork or event, I thought we could perhaps start with a brief outline of how you arrived 
at the concept? 



Erin: I think the concept has been lurking in the sidelines of my practice for some time. It 
began to take form around questions of interactivity, particularly around technologically 
innovative art projects that themselves question how art tackles notions of participation. 
Two issues seemed most salient for me in this turn toward the technological: 1. How do we 
not become too entranced by the technology itself, bending to its needs — how, as artists, 
do we not fall prey to feeling as though it is technology that provides the experience. Or, 
put differently, how do we not fall prey to the idea that it is technology that supplies the 
wonder, while at the same time not dismissing the complexity of technology and the many 
roles it can play within our practices? 2. How do we retain a sensitivity to the art- event (not 
just the technology-event), keeping in mind the difference between interactivity and relation, 
between the setting up of a cause-effect scenario and the creation of an event. 



These questions led me to take the process of investing in digital technologies very carefully, 
wanting to be certain that I could back out at any moment. My sense is that once the 
investment in a technological process becomes too dominant, we can lose sight of the field 
effect we are looking for — an effect that may be available with much more limited use of 
technology. This is not to speak against the use of technology, but to ask how technology 
itself becomes artful. How to create a patient investment in "what the art can do" and not 
just "what the technology can do." 



"Entertaining the environment" comes out of this thinking. It is a reminder not to place 
ourselves too quickly at the center of each experience. It suggests that what is perceptible 
may not be immediately available to us, or may be obfuscated by our expectation that 
relation always includes us. And it perhaps pushes us to reconsider how experience unfolds, 
leading toward more nuanced interpretations of what participation can mean. 



"Entertaining the Environment" also places us immediately in a relational framework rather 
than investing in the hierarchy of subject and object (human and nonhuman). When the 
human is considered the centre of the experience, the sense is that the entertainment 
also has to fit into human-scales of time. In an art experience, this usually means that 
the access to the artwork has to be quite quick — the attention of the spectator must be 
secured within seconds. But when it's the environment that is being entertained, suddenly 
there is a different sense of duration. It is not solely about us, but about how the various 
assemblages — concrete and abstract, human and nonhuman — are realigned through the 
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artistic process. Concretely, this means that we begin to design, or better to create platforms 
of relation, for more ephemeral participants — air currents, movement, breath. And in doing 
so, we are perhaps more aware of how space is crafted, how time itself is artful. 




Weather Patterns — Erin Manning, Nathaniel Stern, Bryan 
Cera, Andrew Goodman, 2012 



Andrew: This is going in several interesting directions already ... Perhaps to bring it back 
to your first statements about technology, there does seem to be a general difficulty 
in finding a balance where technologies are utilized in art works. So many works seem 
invested in a demonstration of the technology's capabilities (and/or the artist's technological 
skills). Likewise in 'interactive' work (a problematic term at best), there is a tendency 
to demonstrate the interactions/ connections on a very overt level — a doubly deathly 
combination when interactivity and technology are combined. Somehow both artists and, I 
think, viewers need to get beyond the entrancement with what the technology is doing and, 
as you say, back to "what the art can do". If we think of painting, for example, I don't think 
anyone would accept that the major conversation between a painting and a viewer would 
be about the pigment or type of medium used, even if the painter or a painter/viewer might 
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be invested in thinking this through. And in fact when we watch TV or go to a movie, for the 
most part the huge technological complexities that allow such events to happen are hidden 
from sight — even CGI imagery needs to do more than demonstrate power nowadays to hold 
an audience — we want a different kind of engagement. It seems naive for an artist to think 
that they could supply much wonder through technological demonstration, considering the 
capabilities of Xboxes/iPads and so on. 



Given all that, your approach of investing cautiously in technology seems a wise tactic. I try 
to remember the relational works of Lygia Clark as a benchmark of what might be achieved 
through very simple means. Perhaps we should all plaster our studios with images of her 
work, along with Dan Graham, Robert Irwin and Steina and Woody Vasulka to remind us of 
the imaginative possiblities at the fringes of technology. At the same time, electricity, for 
example, seems to have interesting possibilities in terms of thinking forces outside of any 
human agency. I'm thinking of the earliest experiments/art events with electricity — running 
a current through a line of 300 monks holding hands, for example (it's the image of monks 
holding hands that brings in the poetry). [2] While clearly this has a 'demonstrative' element, 
it seems to me also involved in a shift in positioning the human in the environment — an 
understanding of and entrancement with environmental forces capable of transversing and 
reorganising the human. In this way perhaps technology does open possibilities for thinking 
art events outside of human-centric fields. 



Erin: Andrew, I love this image of the monks — particularly when we think of it less as a 
human circle than as an electric circle activated through a collective body. Lygia Clark 
is certainly an example I hold on to, particularly as a reminder that the art object is 
not ultimately what art is about. The artfulness of art is about the lure it activates, the 
provocation. A painting is a lure for feeling-seeing texture- become-image or shadow- 
become-sound (to think of the use of calligraphy in early Chinese painting). Lygia Clark's 
relational objects were not "valuable" or "artistic" abstracted from what they could do — 
they were "just" bags, rocks, air. But taken in concert with the relational field they were 
capable of activating, they became-art, became artful in the sense that they were capable of 
affecting the environment they were co-creating. 



Technologies, as you point out, are ever-present. We cannot conceive of a world without 
them, nor should we. The point is to activate them at the level of their integration into a lure 
that stimulates the event, not to make them the event in their own right. It's not that I don't 
think technology-in-itself can't be an event. It's just that I don't think that is the best use 
of an artist's talents. Microsoft, NASA, Nike can make technology an event — they have the 
means to do so, and their teams are poised to produce the newest-new. Art, it seems to me, 
is best at doing something different: at making apparent the interstices between capitalisms 
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and their outdoings, at making apparent the interstices between the present and the folds of 
time that run through it. I think of art as proposing an event-time that is not allied to linear 
time, that is not about novelty per se, but about creating alternate conditions for a tweaking 
of experience. 



Andrew: I like the fact that we're including painting in this conversation. It seems to me that 
too often its relational possibilities are overlooked in favour, once again, of mediums more 
overtly able to demonstrate relation, whereas really any mode of art has potential to include 
interesting events of relation, as it can also fall into representational traps. 




Weather Patterns — Erin Manning, Nathaniel Stern, 
Bryan Cera, Andrew Goodman, 2012 
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What we are talking about here could be defined as the making of 'propositional' artworks — 
Whitehead's definition of a proposition being of a 'lure towards feeling'. This most obviously 
links in Western art history to conceptual art, but also whenever events of relation are 
thought of as the primary artistic 'product', whether between objects (Duchamp's Three 
standard stoppages [1913-1914]), objects and bodies (Clark's Caminhando [1963]), or purely 
the conceptual (Yoko Ono's Grapefruit [1964]). I mean that it doesn't exclude the making 
of objects, but that they are employed tactically rather than representationally, valued for 
their ability to condition, to seed the actualisation of interesting events. It does seem to me 
though, that there is perhaps an interesting shift from much of 60s/70s conceptual art to 
contemporary propositional works such as yours — a shift away from the index and towards a 
concern with the much more slippery areas of affect and sensation. It relates to conceptual 
art in that it is concerned with an open-ended 'thinking-through' of concepts through action 
and is not about representation, but the events produced are less concerned with activating 
conceptual processes in the viewer/participant, and more with activating Deleuze's 'blocs of 
sensation' (my favourite definition of art). 




Weather Patterns — Erin Manning, Nathaniel 
Stern, Bryan Cera, Andrew Goodman, 2012 
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Paul Klee defines art's purpose as making things visible, making us see or experience 
in a new way, which I think fits in with your statement above about art not being about 
novelty but rather allowing a 'tweaking of experience' — reconfiguring old or accessing new, 
and potentially decentered, relations within the world. Perhaps this brings us back to the 
question of what reconfigurations of relation/experience "Entertaining the environment" 
might specifically offer? One of the first things that comes to mind for me, suggested by the 
title of your work in the exhibition — Weather patterns — is an interest in reconnecting with or 
embracing the forces of multiplicities within nature [3]. Michel Serres refers to multiplicities 
as 'nebulous set(s)... whose exact definition escapes us, and whose local movements are 
beyond observation' (1995: 103), and he lists heat, flame, clouds, wind, and climate as 
instances of multiplicities with transformative powers that 'nature makes us live in'. Are 
connections with these kinds of 'unknowable' fields of relation of interest to you in your 
work? 



Erin: Absolutely! I am thinking of weather as that which surprises and disrupts, and also that 
which is absolutely everyday, backgrounded from experience. Whitehead talks of negative 
prehension, referring to aspects of experience which actively make up experience without 
being prehended as such. I think that for the most part this describes weather. Though, 
in countries like Canada (and perhaps, with climate change, more and more countries are 
going to move in a similar direction), weather is also that which explicitly moulds experience. 
It is not simply that which is expected, it is that which is overcome (be it the -30 of winter 
or the +30 of summer). This would also be the case in places that high rates of floods or 
tornados, or for farmers who depend on weather for the crops. In such cases, weather 
itself becomes propositional, an activity that not only frames but also creates modes of 
engagement.) 



Weather Patterns as a piece plays on all of this, but with a focus more on the side of 
negative prehension. I think of it as a weather system in its own right — a sound-and-wind- 
maker that responds not only to your direct interaction with it, but to the multitudinous 
electromagnetic variations in its wireless field. The idea of backgrounding human interaction 
(or at least not foregrounding it) was based to some degree on weather itself, which is very 
much out of our hands! 



The last iteration of the work (May 2012, Millwaukee USA), with Nathaniel Stern and Bryan 
Cera, complexified the field of interaction by building in a system of digital-analog speakers 
that move the sound through the fabric-field (a line of 45 speakers was created with sound 
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bouncing from one to another). We also created a fan-line that is similarly activated by the 
movements in the field. The data stream itself is activated by sensors sewn into some of 
the fabric pieces (which also have conductive fabric sewn into them). But the focus for me 
is not so much on the technical aspects as on the ways in which this system can make felt 
some of the complexity of weather all the while emphasizing its non-human- centred focus. 



With your collaboration for the next iteration (August-November 2012, Melbourne 
Australia), I see us complexifying the soundscape, which at the moment is very basic. 
Sound is something you have worked with a lot, perhaps you have ideas about how 
sound can best work in a work that seeks to make felt field effects? I know your own work 
has played with these kinds of ideas as well. One of the ideas you mentioned was the 
possibility of making a (sound) effect that is itself negatively prehended — a sound, perhaps, 




Weather Patterns — Erin Manning, Nathaniel Stern, Bryan 
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that unfolds in a time that is not of the human. What would a sound be like that took three 
months to unfold? (I am thinking of the three-month span of our exhibition/collaboration 
project). 



Andrew: So a negatively prehended sound would perhaps have to be conceived as one that 
one (as a human) could somehow become aware of, in its existence, but unable perhaps to 
perceive it — one to grasps it conceptually only. If you take the pitch of a sound outside of 
a perceivable human range — higher or lower — I think there can still be an affectual relation 
to the body: low sounds experienced as some kind of almost rhythm or pulse in your bones, 
and high sounds that are almost felt as a sensation on the hairs on your skin — that's at least 
my approximation, since they escape any direct conceptualisation — you know them only 
sideways, through their effects. Of course with the high-pitched sounds you 'know' them 
through the effects seen in the environment — most specifically all the dogs start to howl in 
my neighbourhood when I'm mixing. 



To me this could lead into the idea of 'micro-perceptions', things noticeable through affect 
and sensation rather than perception, that as a multiplicity perhaps can become a perceived 
sound. Its something I've been experimenting with, layering eight or more sounds 'behind' 
the dominant sound so that while you cannot ever name them as separate things, they 
add qualitatively to the overall effect. That is, when you take them out it sounds different 
somehow, but the change is nothing quantifiable, almost, I want to say, an affectual 
tonality, that works through the body in ways other than the ears. Perhaps this relates to 
synesthesia — we have to start thinking outside 'normative' perception and about what a 
sound feels like on the skin, what it tastes like, what it looks like, as much as what it sounds 
like. 



But more generally, as you suggest, thinking imaginatively through specifically non human 
time spans and/or fields of environmental forces that other 'beings' can connect with is 
an interesting angle, inherently decentering the human. If we accept from Whitehead that 
all entities are capable of prehension then we will want to specifically think inanimate as 
well as animate and sentient beings — which is where imagination comes in. What forces 
in nature is a rock attuned to — heat, wind, acidity? Where do a tree's sympathies lie — with 
rain, daylight patterns, symbiotic conversations with bacteria? On some level we can I guess 
imagine these things conceptually if not bodily — we can also know mechanically but never 
empathically understand what the changes in sap flow as the days lengthen feels like and 
how this connects a tree to the tilt of the earth. 
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But what about ways of experiencing that we can't even really begin to imagine or name? In 
The embodied mind Francisco Varela talks about different mechanisms for seeing — humans 
have, apparently, developed three differential categories (hue, saturation and tone), while 
some animals have only two, and others have perhaps four or more. These added qualities 
are not simply extensions of our ways of seeing (being able to see infra red, for example), 
but completely new categories. For example, he postulates that there might also be, for 
some creatures, a rhythmic pulse to objects that gives a whole new dimension to 'seeing' 
(Varela, Thompson and Rosch, 1992: 147-180). Even as we know and can already experience 
that the senses are synesthetic and already irretrievably intertwined, this seems to go 
beyond that to truly unknowable forces. 



The big question for me is whether we can manage to make something felt that is so outside 
of human timespan or perception so that it can only be understood negatively. Can this be 
more than a conceptual understanding? That is, can we move beyond a level of pitching a 
tone that humans can't hear, while telling them it exists so that they can conceptualise their 
lack of perception, to a true prehension, related to/in a bodily, affectual or sensual manner? 
I'm not sure if this will be a productive line of inquiry, whether it could prove enlightening 
or too negative towards, not only human subjectivities, but also more-than-human bodies 
rather than establishing new and interesting relations with them ... 



Erin: The challenge, as you say, is not presuming to know how a more-than- human ecology 
makes itself felt not only beyond the human, but also for the human. It would be a relatively 
straightforward move to create a theoretical problem that translated to one that we call 
negatively prehended (that is, work with sounds that are outside of human hearing but heard 
by animals). But this might simply keep us in a standstill as regards experimenting with the 
idea of entertaining the environment — it might presume we know what that means and can 
orchestrate it. It seems to me that the call must remain experimental, that entertainment 
is something that we need to be reinventing all the while. Brian Massumi and I recently 
went back to Whitehead's two perceptual categories "causal efficacy" and "presentational 
immediacy" and rethought them in terms of entrainment and entertainment. We did this 
to try to activate the sense in "causal efficacy" of there being a force that exceeds any 
straightforward notion of causality. As we understand it, the first phase of perception - 
what Whitehead calls causal efficacy - involves an immanently relational intertwining of 
perception with action. It is causal in the sense that it directly activates a field of relation. 
It entrains. And out of this entrainment follows the possiblity of the activation (the self- 
activation, at the level of the field itself) of a notion of entertainment, or what Whitehead 
calls "presentational immediacy." Entertainment here is not about the human being 
entertained by the environment, but about the direct perception of the fielding of experience 
such that it brings its qualitative resonances to the fore. I think this is what we are talking 
about in terms of "entertaining the environment." We are not wanting to explore the idea of 
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an instrumental, human-centred approach that involves "entertaining" the environment. That 
would just take us back to square one. Instead, we are asking what it might look like, feel 
like, be like, for entertainment to be given back to us as a field of relation. 
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between experience, thought and politics in a transdisciplinary framework moving between 
dance and new technology, the political and micropolitics of sensation, performance art, 
and the current convergence of cinema, animation and new media. Publications include 
Relationscapes: Movement, Art, Philosophy (Cambridge, Mass.: MIT Press, 2009), Politics of 
Touch: Sense, Movement, Sovereignty (Minneapolis: Minnesota University Press, 2007) and 
Ephemeral Territories: Representing Nation, Home and Identity in Canada (Minneapolis: 
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Notes 



[1] Entertaining the Environment was an exhibition in Melbourne in during 2012. See < 
http://www.andrewaoodman.com.au/388/ >. For more on the concept of entertaining the 
environment, see Manning, 2011. Weather Patterns is a work by Erin Manning, Nathaniel 
Stern, Bryan Cera, Andrew Goodman, exhibited in one iteration as part of Entertaining the 
Environment. 

[2] See Elsenaar and Scha, 2002: 19. 
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[3] For more on Weather Patterns, see http://www.erinmovement.com/erin manning 
weatherpatterns exhibit.swf . 
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Amsterdam School for the Arts. 

Andrew Murphie. 

University of New South Wales. 



Andrew Murphie: I am interested in your background, in how you see your own movement 
through the use of media. Something I find really interesting is your work with the translocal 
and the collaborative. I'm also interested in why you are thinking about moving this towards a 
more directly material practice. [1] 

(The online version of this article includes the embedded video Interfacing/Radiotopia/ 
KeyWorx (DEAF03) which can be accessed at http://www.v2.nl/events/interfacing-radiotopia- 
keyworx. -ed.) 



Sher Doruff: Yes! (laughs) Caught in a transitional vacuum! Would you want a little bit of a 
history? Is that helpful? 



Andrew: Yes, that would be fantastic. 



Sher: I went to art school from '68 to '72. I was attracted to what was then emerging, 
which was conceptual art. But I was unhappy with my art department. So I switched to the 
philosophy department. In my junior year a new faculty was hired in the art department and 
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they confronted me and said, 'Why do you want to be in the philosophy department writing 
papers? You can do-what-you're-doing in the art department, and it's just much more fun!'. 
And it was actually a very silly, but convincing, argument! (laughs). So I thought, 'I'd really 
rather make things than write papers'. So I transferred back to art. 



I was reading Whitehead's Science in the Modern World at that time which had a 
tremendous impact on me. I was nineteen. I'm sure I didn't understand much of what I 
was reading really. I remember having an elaborate plan for a kinetic sculpture based 
on Whitehead's theory of duration (laughs), and if I think about it now it was probably 
very clunky and not-very-well conceived. But I was very excited about it. I worked on the 
sketches and when I finally finished my plans, I was quite excited about this piece. I figured 
it would take me two months to build it. When my mentor looked at my plans he said, 
'You're done!'. 'What's next?'. And I thought, 'What do you mean, I'm done? I haven't even 
started making it!'. He said, 'There's no point in making it, because as far as I can tell the 
work has been in your process of thinking this through, and then you would be spending two 
months making this thing in which there would be some skills you would learn, but I don't 
see the point in it.'. And that was completely confrontational. I still don't know if it was good 
advice, but it was advice that — it's something that has resonated my entire life. 



Andrew: And you still want to work with diagrams! 



Sher: (Laughs) Yes! It's a frustration. Anyway. When I left art school in '72, leaning towards 
conceptual art, I had no idea what to do with it! (laughs) So I was in a band for ten years [2]. 



Andrew: Really? Ten years? 



Sher: With the same people. This is where I learned about collaboration. We rehearsed 
every day we didn't gig. So really, in my twenties, in the seventies, what I learned, and what 
excited me was that you come together with a group of people and you make something 
happen, and you figure out how to do that together. So that's my history. Eventually I 
became a multimedia artist in New York, working mostly with dance and theatre companies. 
I've been working with computers since '85. It became my tool of choice, and I worked 
mostly with sound and then with graphics, and then picked up video when the quicktime 
protocol was launched in the early 90s. 
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KeyWorx projection from the Cassis Caput/Helmet Head translocal public webcam 
dance performance (Nora Heilmann/Sher Doruff) 2003. From left clockwise: 
Amsterdam, Berlin, New York, London, (image provided by Sher Doruff) 



Andrew: So you would have been involved with the initial transformations in dance involving 
multimedia? 



Sher: Yes, in the eighties/nineties... it was very experimental and there was a lot of leeway 
for failure, and I think that in that sense it was very exciting ... 



In 1990 my apartment burned down and I travelled for awhile. I got a residency in Paris for 
a year, ending up in Amsterdam and eventually in residence at STEIM [3] where I met Tom 
Demeyer [4], who was working with Steina Vasulka [5]. Steina was Visiting Art Director of 
STEIM at that time, and she was working with Tom on a programme called Image/ine [6], 
and Image/ine ... as far as I know, was the first personal PC application in which you could 
effectively use real-time digital video editing. 
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Andrew: I've seen Image/ine. 



Sher: Yes. I think it really was the first. Tom was, in my opinion, a genius programmer, and 
to have Steina as a collaborator, well, it was magic. Really magic. I'd taken some courses in 
digital video editing and AVIDs in New York and I learned quite a bit about digital filtering. 
When I saw Tom coding it in real time on a Mac desktop computer, it was phenomenal to 
me. It was also so aesthetic because it had many of Steina's ideas in it. So I spent a great 
deal of time nosing around the computer lab, checking out what they were up to. 



Tom eventually asked me to write the technical manual for Image/ine. I'd never written 
a technical manual before and I thought, 'Can I do this?'. I threw myself into the work, 
and spent months trying to put together an informative guide. I learned so much from 
that exercise because it entailed discovering and explaining every connection, every 
interconnective relation of every parameter to every other parameter of the media objects 
in the application. I went deeply into the interactive possibilities and that was really 
transformative for me. 



So that was my entrance into interactive dynamic media. This is 1996/97. I started using 
Imag/ine and performing with it, and I taught many workshops to performing artists of all 
kinds. Those workshops were really interesting. I had something like twenty, thirty people, 
often, in those workshops, trying to teach them real-time interaction with one desktop 
computer (laughs) ... many of them didn't even use email at that time so the computer 
environment was somewhat frightening to them. But in those workshops we also always 
had some media artists, and one or two programmers mixed in with the performing artists. 
It was an exciting mix because, in fact, the performing artists were more liberated in 
their conceptual use of the technology than the media artists and technicians who were 
somewhat stuck behind their computers experimenting with the software itself, with what 
the software could do, and not looking beyond the code to the relation to, well, whatever 
was happening in the space itself. 



At some point, I think it was 1997, a colleague asked me to come up with a proposal for an 
online virtual studio environment. I'd been working with Image/ine's real-time capacities 
so much that I thought, 'What if there was an online site where you have the ability to 
interconnect media parameters - sound, video, image, text?'. What if it was a collaborative 
open environment in which all the players see the same dynamic interface and make choices 
together? A performative situation in which you would continually enact/react to the choices 
and content of those performing with you. Synchronous interaction. How to enable this 
activity over the internet in '97 was completely blue sky although easy enough to imagine 
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as technological protocols progressed. Keystroke grew out of this original proposal. To have 
any hope of actualising it, it needed Tom's skills and brilliance at real-time processing. 




Jeff Mann, Michelle Teran and Niels Bogaards using KeyWorx site specifically 
in Banff CA during the HotWired LiveArt2 workshop in 2001 
(Image provided by Sher Doruff) 



So, yeah, imagining that some day the internet — as an enabling environment — could be a 
zone of processing video, images, sound and text in real-time. In the late ninties ... the only 
other collaborative environment was Resrock, as I recall, and that was only MIDI enabled. 
Using the MIDI protocol, musicians could send signals back and forth building musical 
compositions in real time. MIDI was fast enough for broadband, for modem speeds at that 
time. So we wrote a proposal with the Waag Society for Old and New Media in Amsterdam 
as our support institution. We received eighty thousand Guilders — that's roughly forty 
thousand Euros. It was enough to give the project a kick start. 



Andrew: So it's still continuing? 



Sher: Yeah, it's changed names and direction but yes, it's been KeyWorx since 2003ish. [7] 
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KeyWorx screen shot sequence from the performance by 
Michelle Teran in Rotterdam and Isabelle Jenniches in New 
York. Taken from Connected! LiveArt (Doruff, 2005: 42) 

We began in '98. Tom came on board with Just van den Broecke, who had worked with 
Lucent and Bell Labs on multi-location and multi-user experiments. He was the perfect server- 
side programmer for this project. We were lucky to find Niels Bogaards, who began as an 
intern, to work with me on the interface. So it was a very small team originally, working one 
day a week on a project that was admittedly crazily ambitious. But over time we actually 
managed to produce something promising. Lodewijk Loos and Eric Redlinger were important 
contributors. As we progressed, bandwidth improved in parallel. Our progress seemed to 
follow the technology protocols in such a way that, by 2000, we had a very, very buggy 
application that worked well enough to excite a small group of dedicated artist practitioners. 



[...] 



Sher: I had learned to program MAX in 1990 and that encounter with modular interfacing was 
influential. We wanted the Keystroke interface to be modular but not as spaghetti-messy. In a 
collaborative environment, there's no way a MAX patch can be effective when other (multiple) 
people are simultaneously working with it. It's just too — it's too chaotic and too idiosyncratic. 
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We wanted to have the same kind of modularity in an interface but much more intuitively 
rendered. People would need to enter and somehow easily understand what was happening 
in a given patch. 



But also, and this was unique to Keystroke, parameters of every media object and controller 
would be transparently open so that, for example, I could change the speed and position 
parameters of one of your movie clips with the frequency and amplitude of one my sound 
clips. That sort of activity opened across a full range of devices and objects. So basically you 
needed to adapt to creative negotiation on every level. It's collaborative like a theatre piece 
is collaborative yet more so. You could enter this environment and say, 'You're working with 
visuals, you're working with Quicktime movies, with moving imagery, with still imagery, or 
perhaps someone enters with sound material'. You might decide together to assign roles 
and functions, but then you might not. All media and controllers (mouse, camera, joystick, 
GPS, etc.) were objects in an open field. The thing is, if you have a modular application your 
'playing field' emerges from the negotiations of the players and the transformative objects in 
play. 



Andrew: So it's a modular system but you have all these interstices where negotiation can 
arise. I should say that before we started the interview we were talking about coming out 
of theatre and arts backgrounds, and how you learn a lot more about collaboration in these 
areas. You really do have to negotiate with people, and you also have to go somewhere with 
them, and I guess, finish the work with them. Get some satisfaction, shall we say, of some 
kind! Even if it results in death! So what you're trying to design ... it's a kind of an open 
framework. You're designing for potential collaborations rather than for those that are really 
heavily structured from the beginning. 



Sher: Right! It's essentially about conditions — we tried to limit conditions in the application 
design as much as possible ... the conditions are established by the players and emerge 
from the play. KeyStroke/KeyWorx provided the tools, the basic components. The conditions 
of play can be established while playing or before arriving to play. A lot of the people that 
use the software — I mean, there was never anything normative in who used it and how. 
Some people couldn't work with the potential chaos of emerging conditions so they tended 
towards structured improvisations where they would find some way to establish the initial 
conditions before they would enter the session, and everyone would have certain terms or 
tasks or goals. Or roles would be distributed between the people who would join. Everything 
would be pre-set before they would even begin. With others, sometimes you could be 
playing in a session and someone else would enter who wasn't even invited, because the 
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server was open. Then all of a sudden you've got somebody new in this field. They might 
not be aware your conditions or style of negotiation. You might also choose to type text 
in real-time, and your negotiations could become part of the performance, because you'd 
be writing to each other and those texts would be 'public'. On the other hand, a lot of 
the people who became more adept at the programme would usually run an IRC chat on 
the side. They'd chat about what they wanted to do next, often not as part of the artistic 
material, of what you might be experiencing as a spectator. 



Andrew: Kind of a meta-dialogue. 

Sher: Yes. But it was often interesting to me to see that negotiation become part of the 
piece itself, you know? If that was the artistic choice. 



Andrew: There is a kind of ongoing meta-modelisation in that. [8] The thing kind of shifts, 
as you say, according to the players, and it forms a kind of genetic modelling as it goes, but 
these models are changing with time — 



Sher: Right — 



Andrew: And there's layers of that going on, as well. What were people doing? What were 
some of the results when people were using Keystroke? 



Sher: Well ... there were many things about KeyStroke/KeyWorx that were difficult to 
adapt to because it was always in Beta, and it was always buggy, and we had terrible 
difficulties with network protocols and the emergence of firewalls. So the software itself 
was constantly having to deal with these kinds of issues. Nonetheless, we had a devoted 
contingent of fearless and patient artists. I'd say maybe, back in the early 00's there 
were only twenty or thirty — quite a few interested Norwegians such as Amanda Steggel 
and Per Platou from Motherboard who sponsored the first Keystroke workshop in 2000. 
Also Michelle Teran and Jeff Man, Isabelle Jenniches, Arjen Keesmaat, Nancy Mauro 
Flude, Linda Dement and several others. It was interesting, because many of those first 
performances were between Australia, Canada, Norway, New York and Amsterdam. So 
I would say that those were often the points of contact, the nodes, though they always 
shifted and there were of course performances we would never know about as the app was 
freely downloadable. 
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Andrew: What actually happened? 



Sher: Well, there's a community in New York called SHARE, and they actually became the 
biggest user group through the supportive efforts of Eric Redlinger, and they're still going 
strong. I think they were originally based near St. Mark's Place in New York, and every 
Sunday they would host translocal performances. They used KeyWorx, in tandem with MAX/ 
MSP/Jitter, Pd and other emerging applications. 



Andrew: So you became a kind of religion. 



Sher: Hmmm. I think there were many people ... not a lot of people, actually, to be honest, 
a handful of people, devoted to this idea of translocal, real-time collaborative performance. 
One of the things I try to touch on in my thesis is the notion that translocal experience, as an 
embodied experience, is amplified because your body is ... you have the sensation of ... how 
can I say this? The effect of intensities of translocal performance when it's indeterminate 
and you're collectively negotiating and making choices together and you're playing off those 
choices as a jazz band would or as a dancers might ... the translocal becomes incredibly, 
intensely physicalised, but your feeling of space is non-locally oriented. 



There's almost an imbalance with the "virtual" in a sense. By which I mean the Bergsonian 
or Deleuzian (virtual), not the virtual simulation that we talk about when we speak of media 
technologies and networks. You really find yourself intensely in this differential space. I think 
that this is something that can be found in translocal experience when it is performative and 
not when it's streaming, so that it's passive. That's another experience, and that's not what 
I'm referring to. What I'm interested in is collaborative performance as transducer through 
non-local space. The space becomes ... it's not that it's a non-issue, it's just a completely 
different issue. You end up collaborating and your co-existence ... well, happens between 
your monitors essentially, but in the space of your monitors, as well, and in the shared space 
is this ... 



Andrew: Well it kind of is in the monitors, but its kind of not as well, isn't it? 



Sher: Exactly. Brian (Massumi) has an essay called 'Strange Horizon' (in Massumi, 2002), 
and he talks about the biogram, about topologies, and about vision and proprioception, and 
he suggests, 'Well, maybe even staring into a screen', you have that same biogrammatic 
experience. 
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Andrew: And you used this idea in your discussion of collaboration, I think? 



Sher: Yes. When I read that I thought, 'I know this experience, and I know this experience 
through KeyWorx'. Brian talks about the biogram as the lived experience re-emerging from 
itself. That's what everyone that I talked to who was an avid KeyWorx user says ... people 
like Michelle Teran. She told me she would be in a session for twelve hours and it would be 
only at something like the tenth hour that she would have the sense of losing ... of being 
completely in sync and being in another world. 



Andrew: So, people would get into using KeyWorx for the experience itself? And they would 
do this quite regularly, as a kind of ritual? 



Sher: I believe so. 



Andrew: That's really interesting. I hadn't thought about that aspect of it. 



Sher: Almost every artist said you needed a minimum of three hours in the environment. This 
is three hours of completely focused attention before you could find the rhythms between 
each other, because there's also latency, depending on the network, depending on all kinds 
of network conditions, and also conditions concerning different styles of making between 
the people involved. There are also conditions relative to the different kinds of media that 
you're using and processing, and all of these kinds of things matter, before you find a rhythm 
together. Just as performers also need that kind of time before they find a rhythm together. 
But translocally it's a bit different because you don't have the sensation of the person 
(directly). You can't make eye contact. You don't have perceivable body language between 
you. So you have to find other ways to find that kind of synchronisation. It's incredibly 
intense and affective, and it requires so much focus that it creates a kind of extreme 
experience (laughs). I do think people were in it for that. It was a kind of a high. It was for 
me ... when it wasn't frustrating. I mean, there was a down side to it as well, because the 
technology was never plug and play. You could never count on it not crashing — 



Andrew: Maybe that's part of it. 



Sher: You had to build the idiosyncratic temperament of the network and the application 
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technology itself into your rhythms and into your modes of play. 



Andrew: So when you write about 'polyrhythmic diagrams', [9] is this what you mean? 



Sher: By polyrhythms I just mean ... I'm probably extrapolating a little bit from Bergson 
in that I'm attaching a rhythm to duration, which I don't think he does, but I find that 
interesting because I'm very attracted to his idea of multiplicities of duration. What I find in 
performance is that within those multiplicities, the durations of performers, the durations of 
the 'things' that you're engaged with ... there is an infinite bandwidth of multiplicity, of the 
durations of everything involved in a particular experience. I think in performance it's quite 
easy to associate a rhythm with those durations, and in these kinds of performances, if you 
have rhythms of, let's say, the media components, the rhythms of the processing and the 
thought of the performers and the people who are controlling those processes ... they're all 
differently relating. There are these moments where they're polyrhythmic in the sense of very 
complex and complementary rhythms that sometimes sync, you know, and sometimes come 
together, and you can never really ... you can feel them all at the same time. You can't hear 
them all, because you have to fixate on one or the other. Polyrhythms in Western African 
music are really fantastically complex and you can sort of let yourself go and experience the 
whole, but when you try to listen to them it's almost impossible to hear them all at the same 
time. At least, I have that experience. And that's what I felt happen in these kinds of ... it 
reminded me of Sengalese ... 



Andrew: Drumming? 



Sher: Yes — drumming. But I'm romanticising it! (laughs) 



Andrew: I like romanticising it. As I said, I think the 'not working' and the Beta and 
everything is really part of it. I really do. 



Sher: (laughing) I think if some of the people who used it over the years were here and were 
listening, they would say, 'Oh, yeah, well, that's Sher just going off and riffing.' But taking the 
best parts of it ... this is what it was for me, and I spent a lot of time watching people and 
experiencing what other people were doing. I was so fascinated to observe what happened, 
and how people interacted with this environment, and for me, that kept me going for six, 
seven years! 
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Andrew: Really? So, Keystroke became KeyWorx at a certain point — 



Sher: Yes. 



Andrew: But that's not terribly significant in — 



Sher: No. We had to change the name because eventually somebody had — well, you can 
imagine it was quite a common name. So, only for legal reasons. And it did become open 
source, but eventually Waag Society just couldn't support it anymore. Well, they wanted to 
roll it back and then work on open sourcing it, which they've done, which is great. But at the 
same time, they weren't interested in supporting it as an artist's tool anymore. So now, it is 
a monolithic open source platform mainly utilised by Waag Society programmers for internal 
projects. I'm happy that it's open sourced, but I don't anticipate artists using it any longer — 



Andrew: Really? 



Sher: There's no graphic interface. The initial interface that I designed was very quirky, 
and I still am quite fond of it, but the code quickly outgrew that design. As we kept adding 




KeyWorx workshop at Waag Society, 
(image provided by Sher Doruff) 



fibreculturejournal.org FCJ-153 147 



FCJ-153 Multimedia Mixing and Real-time Collaboration. 



modules the screen real estate just became too limited. We had to rethink this and we never 
had an adequate secondary interface so that people could also easily access a histogram of 
activity in a readable way. But I still think that one of the most interesting things about the 
interface was its openness to intervention — to soft or brutal addition and deletion. I mean, it 
was possible to delete or alter the modules of other performer/players, because the interface 
was openly shared and any device or object entering the interface was shared. 



Andrew: So there wasn't just the power of veto over your own work, it was veto over other 
people's work ... (laughs) This is free cooperation extended plus! (Spehr, 2003) 



Sher: Yes. And often, when people were learning it during workshops, you'd see that 
happening all the time as they were focusing on understanding the operations ... 'Well, 
if I do this ..." and then someone changes your connecting patch before your eyes ... the 
tendency was to think 'I don't want this!' And they'd just delete it! 



Andrew: I'd be terrified! (laughs) Again, it's like the theatre, really, in a way. Those kinds of 
negotiations that we were talking about before: in a theatre, you see it all the time. The 
actors are, in essence, deleting each other's work all the time, because they kind of have to... 



Sher: Overriding ... 



Andrew: In a sense, all that — the Beta states, things not quite working, the deletions, and 
the negotiation of all their theatre and dance and those sorts of ... any kind of collaborative 
(work) involves similar things. 



Do you think something else is coming along now that's going to do — 



Sher: You know, I still haven't seen it. If you look at MAX/MSP/Jitter, or PD — and I feel more 
of an affinity to PD because I would want to support the open source initiative — but they're 
still not shared interfaces. 



Andrew: No. That's right. 
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Sher: These are the technologies that are used in most current translocal performance. 
You're sharing data just by sending data over various network protocols, but you don't have 
that shared dynamic Umwelt. 



Andrew: If it's not a completely shared interface, that's a huge difference. 



Sher: Exactly. And that's the thing that I still haven't seen. Maybe it does exist, I just don't 
know about it. 



Andrew: You know that phrase that Mark Amerika uses?... He writes about the 'asynchronous 
real-time' ... [10] which is a phrase that keeps coming back to me all the time while we talk. 




KeyWorx screen shot sequence from the performance by 
Arjen Keesmaat in Rotterdam and Daniel Vatsky in New 
York. Taken from Connected! LiveArt (Doruff, 2005: 40) 
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Sher: In fact, I think what we mean by "real-time" — because I always question what we 
mean by "real-time" ... 



Andrew: Because in fact the polyrhythmic diagram you're talking about is not quite 
real-time. It's the durational formation of a diagram that's constantly modulating in 
asynchronous real-time — so it is real-time, and it's not, which I think is maybe one reason 
why you get these "highs" working with it. The nature of the duration is really changing, in 
that environment, quite profoundly. I mean you'll never get a high from one moment, there's 
always going to be (a duration of time). That's interesting. 



Sher: Yes, and it was interesting to see with different people how long — there was actually, 
you talk about linear time, there were different stretches of linear time to achieve that. 
Michelle saying twelve hours, most other people saying three, but still, it's ... It's a long 
time. 



Andrew: We were talking last night about the need for patience, and it's true. I think if you 
don't have patience, there's little that happens. It's not just about being "open". You also 
need patience, and you need things to go wrong for a while. 



Sher: But the shared interface, I think it's crucial. It's critical, but it's difficult. It's really 
difficult. And it might be Utopian in a kind of way. Dystopian in another... 



Andrew: It's confronting as well, because people's spaces now are their interfaces. That's 
their space, as well. 



Sher: But on the other hand, we flash forward now, (because I've been blah-blahing 
about all of this ... four years ago). I'm really locked into a 2003 era. In 2006 we now have 
the notorious Web 2.0, we have all of these social networking spaces, spaces of shared 
activity — which I'm really dubious about. 



Andrew: I was going to ask what you thought about that. 



Sher: In fact, I'm just not interested! And I'm not sure why that is, because I've spent 
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so much time thinking about it, and working from these near-utopian visions back in the 
nineties — "Oh, wouldn't this be exciting!" — to the reality of making, and being involved 
in the development of such an application, and the highs and lows of all that, and seeing 
what's possible and what's not possible. What's interesting and what's not interesting. And 
now, in western culture where social networking has become so influential ... "influential" is 
maybe not the right term ... I find myself... I'm not interested in these environments at all. 

Maybe it's just because of my particular history. I overdosed, maybe, on the social ... 




KeyWorx screen shot sequence from the 
performance by Lodewijk Loos in Rotterdam and 
Eric Redlinge in New York. Taken from Connected! 
LiveArt (Doruff, 2005: 38) 



Andrew: You were talking about this before the interview, about your feeling of wanting 
to move back to materials, and you're not quite sure whether that's because you've been 
too involved in technology, whether you've overdosed on technology, in a way. I'm quite 
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interested in where it's leading you. 



Sher: I'm not sure I entirely understand it myself, but I feel it very strongly. I don't know. 
Maybe it's the realisation of having been behind a computer screen for over twenty years, 
and having so much of my life revolve around bits and bytes, in an immaterial sort of way. 
Maybe it's just a natural process. Perhaps I'm merely entering a continuum that says, 'Now 
I need tactility. I want to work with materials again because there's been a dearth of it'. I 
mean — yeah, I don't see myself becoming a Luddite or something like that. It's difficult to 
talk about now, because I haven't really ... I'm at the beginning of something, and I also 
don't want to push it ... 



Andrew: Yes. I don't want to push you too far on it. 



Sher: ... into a rationale. 



Andrew: Let me ask you another way! (laughter) With KeyWorx, it sounds almost like it 
reached its own kind of 'self-enjoyment' or 'satisfaction' at a certain point ... [11] 



Sher: Yes. 



Andrew: In a collaborative sense, actually, there was a larger social event, a larger "occasion 
of experience", whatever you want to call it. There was this actual entity there, a 'KeyWorx 
experience', right? But it perhaps reached a point and that was it, it was there and it was 
actualised, then, after ... in some ways it seems to be ... not gone for everyone, but people 
are picking up on that in different ways. I guess, to think more positively, in terms of what 
you're thinking of moving to now, what are the kind of 'prehensions' being drawn from the 
KeyWorx experience? The things that are coming out of that that are leading you towards 
the more material approach? It's not like you're rejecting the technology. It's like, OK, that 
experience has finished, but what's coming out of that experience that's bringing you back to 
materials? Again, I don't want to push you, because I know this is an initial period, and you 
don't know what it is... 



Sher: These are my questions as well. I've always been interested in process, and product, 
for me, has always been a by-product of process. 
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Andrew: Yet something else takes off. 



Sher: Something else takes off. And I've come to accept that that's how I make work, that's 
what I do, and it's not about these final, finished assemblages that can be aestheticised in 
some sort of way. There's something about an emergent aesthetic that is very ephemeral, 
that I've always been after, and what touches me and what excites me. And I think I've been 
involved in processual aesthetics with digital media now for however many years, and I'm 
interested to see what kind of format that takes when I'm using other kinds of materials 
other than bits and bytes. And it may be complementary — I may be using digital media with 
the material. I just don't know. These things I have yet to find out. But what I'm pretty sure of 
is that the idea, my attraction to processual aesthetics is my bottom line. 



Andrew: OK. And so — it's not just processual though, is it. You want to set up kind of complex 
forms of collaboration and negotiation, from which interesting things are going to emerge. 



Sher: Yes. But I'm also open to the fact that maybe there will be some kind of artefact now! 
(laughs) So I'm also wondering what that might be. How I work with processual aesthetics, 
how that might have matured into something that might in fact become artefact. 



Andrew: This all sounds very different to where institutions pre-territorialise the whole thing, 
in a way, so it's already determined. Pre-territorialisation just kills it right from the beginning. 
I think that's a very common schema coming up. There's that great book by Jean-Pierre 
Dupuy where he talks about the 'mechanisation of the mind' (2000). He's knows cognitive 
science very well. Yet he almost doesn't really believe in "cognition", as far as I can tell. He 
thinks the cyberneticists were wrong, but he says at least they were interesting and complex! 
Whereas he writes that cognitive psychology, when it came out, it really just got rid of all the 
complexity and the possibility of emergence, and it threw away the polyrhythmic diagrams 
and non-linearity and just went for a linear, simplistic version of things. He says it killed 
everything. Everything becomes pre-packaged, pre-territorialised, which is what a lot of 
bureaucrats like. 



... I'd like to hear more about your actual work in the mid-nineties. You've referred to this as 
being another story. 



Sher: Well, it depends on when in the mid-nineties. In the early nineties I was still working 
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mainly with sound composition and what was then called multimedia. 



I found that, around '95/'96 Image/lne (real-time interaction) was what I needed. I had 
run into a wall with sound composition, because I had locked myself into a genre I wasn't 
comfortable with, and I wasn't finding openings for myself. When I had the possibility to 
watch this new interactive platform in its inception, and follow it on a day-by-day basis, 
I found it quite exciting that there was an openness and an interconnectivity between 
parameters of any kind of media — well, that wasn't quite true, necessarily. In the beginning 
it was mainly video processing, I had always admired the Vasulkas and admired Steina's 
work, so it was interesting to work with a new application that I could use as well. It was an 
exciting time, and yes, it's hard to relive, because now we're so used to these capabilities 
that we don't give it another thought, but back in the mid-nineties real-time filtering and 
real-time processing without a stultifying latency was just so fucking amazing! 

Andrew: I know what you mean. I just discovered it in 2006, of course, but ... I'm about ten 
years behind. 



Sher: (laughs) And I'd worked in the theatre with slide projectors and with 16 mm film 
and overhead projectors in New York. Suddenly, you know, access to digital technology 
expanded possibilities within theatre and dance contexts. All live performative contexts. At 
that time I was not necessarily interested in authoring my own work, but found collaborating 
with the directors and choreographers more to my taste. 



Andrew: OK. This is something I hadn't understood. 



Sher: In Amsterdam, for example, in the late nineties, I was sometimes commissioned by 
composers or ensembles to create visuals for their performances ... sometimes utilising 
interactive techniques. 



Andrew: There's a real friction, you've said, between the experience of the collaboration — 



Sher: The experience of the collaborators and the experience of the audience? 
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Andrew: Yes. 



Sher: This is something quite important to keep in mind, because everything that I was 
talking about in positive and glowing terms about the experience of the event was from the 
perspective of the interrelation between the performers. It's quite a different experience for 
an audience, and I think that remains problematic, because the shared intensity between the 
performers is strategically focused and concentrated. The audience experience is often very 
different. But this is another major topic ... 



Postscript 

Andrew: So, looking back on this conversation, which took place in 2006, where have things 
gone for you? Have you created some artefacts? Or did translocal technologies keep their 
interest for you? 



Sher: Looking back on this conversation I'm surprised by the unabashed enthusiasm of my 
offhand remarks. It must have been the lovely sunny day, good company and very strong 
cup of coffee that fueled my exuberance in the re-telling of the good ole days of the dawn 
of interactive digital media. It's strangely unnerving, in a good way, to be reflecting on this 
conversation today. Were we to meet over coffee tomorrow the conversation would no doubt 
be animated with a quizzical critique of the medium that once seemed poised, from my 
perspective, to evoke a paradigmatic shift in the arts. 



It's not as though I view this 'genre' as failed, it just did not sustain my interest. Perhaps its 
pioneering relevance exhausted itself for me. Clearly, the advent of what is now pervasive 
streaming media, Skyping, and social networking has subsumed the translocal experience 
into an indistinguishable mix of synchronous and a-synchronous communication. That once 
magic flutter of passing through potential itself escapes me now when online. 



The coming of the Web 2.0 revolution which was briefly alluded to here in '06 indeed effects 
my everyday life. I no longer struggle with bugs and crashes in daily doses but then I no 
longer experiment with alpha and beta hardware and software applications. I rarely if ever 
use social networks. 
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Is this the worn old script of the suspicious senior who now tires of keeping pace with the 
new and romanticises the frontier past? I hope not although I do think the 80-s-00's were a 
rarefied coming of age of digital arts. I have a persistent sense that the digital lost the fight 
to de-instrumentalise. It now effects just about every medium as a tool but has, arguably, 
dissipated its affectivity as a material. 

You ask Andrew if I have created artefacts since we spoke. I would have to say, no, not 
really. I have spent some years researching diagrammatic praxis which continues to insist 
on de-territorialising incipient form. There is an affective tonality of the KeyStroke/KeyWorx 
experience still at play I suppose. This is perhaps at the heart of my activities now as I 
mainly teach, tutor and supervise artists in Artistic Research programmes. What I can say 
regarding the resonance of the 20+ year involvement in interactive media is that I can 
only approach academic guidance collaboratively and from a beginner's mind. All those 
years of linking and unlinking the dynamic parameters of things/objects/concepts affects 
the way I encounter theory/practice relations with students. It's a learned attunement with 
the fluctuating intensities of the non-relation of relation, as Brian Massumi might say, that 
somehow describes the reciprocity of discovery I feel with students. So the field of play has 
shifted, but the collaborative sensibility is as strong as ever. 
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Andrew Murphie is Editor of the Fibreculture Journal (digital media + networks + 
transdisciplinary critique), and Associate Professor in the School of the Arts and Media at 
UNSW. Andrew's research examines the productive nature of differential intensity. He works 
on transformation, crisis and possibility — as these are filtered through generative process in 
media, arts and philosophy, dynamic modeling of all types, and new forms of cooperation in 
politics/social organization. He is currently writing a book — Differential Media, Differential 
Life: the past and future of social organization — that rethinks the 'world as medium'. 
Andrew's work also draws on electronic arts and design (eg cross signal processing), 
poststructuralism (Deleuze), process philosophy (Whitehead), 'speculative pragmatics' 
(Massumi/Manning), and extended and dynamicist theories of mind. He also works on the 
new publishing (academic publishing to books, music, journalism), and related events in 
education and knowledge mobilization and exchange (technics, methods, and new network, 
information and attentional literacies), [ http://www.and rewmurphie.org/1 



Notes 



1. This interview took place in 2006, in Montreal, during the workshop Dancing the Virtual at 
Senselab. Many thanks to Erin Manning and Senselab. It was edited, updated and extended 
in 2012. Thanks to Liz Brownlee and Xavier Fijac for help with the transcription. The 
interview was funded by the Australian Research Council as part of an ARC Discovery grant 
concerning "dynamic media", on which Murphie worked in partnership with Anna Munster, 
Adrian Mackenzie, Brian Massumi, Mat Wall-Smith and others. 



2. Although, as discussed, Doruff was in an underground band in Chicago in the 1970s, Care 
of the Cow. See < http://careofthecow.wordpress.com/ >. 



3. For STEIM, see < http://steim.org/ >. 



4. Tom Demeyer is now Head of Techology and of the Future Internet Lab at the Waag 
Society in Amsterdam. See < http://www.voutu be.com/watch ?v=T5k4Kval-Qc >. Here 
Demeyer compares 'data with clay. It is something which a programmer can get creative 
with'. 



5. On Steina and Woody Vasulka, see < http://www.vasu I ka.org/ > , with a biography of Steina 
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at < http://www.vasulka.org/Steina/Steina bio.html >. 



6. On the development of Image/ine see < http://timeline.1904.cc/tiki-index. 
php?page=lmage/ine > and Steina Vasulka discussing her perspective at <http://vimeo. 
com/41 196405>. The Image/ine site is at <http://image-ine.org/index.html>. 



7. Keystroke became KeyWorx in 2003, 'due to trademark issues' ( http://www.kevworx.org/) . 
It was developed by Waag Society, Amsterdam ( http://waag.org/en/search/node/keyworx ). 
For more on KeyWorx see Doruff, 2005. See also the Sensing Presence page, at the Waag 
Society, where Doruff, 2005, among other publications, can be downloaded < http:// 
spresearch.waaq.org/papers.html >. 



8. On metamodelisation, see Genosko and Murphie, 2008; Watson, 2008; Wall-Smith, 2008. 



9. On the translocal and polyrhythmic diagrams, see Doruff, 2006a and 2006b. There is 
more material on her 'Diagrammatic Praxis' site at < http://www.researchcatalogue.net/ 
view/? weave =6622 >. 



10. 'By asynchronous realtime I am referring to what at times feels like a perpetual jet- 
lag consciousness or timeless time, a blur motion of experiential metadata that indicates 
a formal investigation of complex event processing where the VJ artist, always gyrating 
at a pivotal location in the narrative, becomes a multitude of flux identities nomadically 
circulating within the net-worked space of flows (both geophysical networks and cyberspace 
networks). Living in asynchronous realtime often produces a feeling of being both avant- 
garde (ahead of one's time) and time-delayed or even preempted' (Amerika, 2009: 26). 



11. 'Self-enjoyment' and 'satisfaction' are Whitehead's terms, for the very process of life, 
drawing things together into an 'actual occasion'. Crucially, once this occasion has reached 
a 'satisfaction', it is finished, though becomes available for further 'prehensions' into new 
'actual occasions' or 'occasions of experience'. In this there a — 



. . .certain immediate individuality, which is a complex process of appropriating into a 
unity of existence the many data presented as relevant by the physical processes of 
nature. Life implies the absolute, individual self-enjoyment arising out of this process 
of appropriation. I have, in my recent writings, used the prehension to express this 
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process of appropriation. Also I have termed each individual act of immediate self- 
enjoyment an occasion of experience. I hold that these unities of existence, these 
occasions of experience, are the really real things which in their collective unity 
compose the evolving universe, ever plunging into creative advance. (Whitehead, 
1938: 150-151) 



On Whitehead and technical interaction, see Murphie, 2005. 
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